UNIVERSITY  OF  SOUTHERN  CALIFORNIA 


ESTIMATION  AND  DETECTION  OF  IMAGES 


DEGRADED  BY  FLuM- GRAIN  NOISE 


Firouz  Naderi 


Image  Processing  Institute 
University  of  Southern  California 
University  Park 
Los  Angeles,  California  90007 


Sponsored  by 

Advanced  Research  Projects  Agency 
Contract  No.  F-336 15-76-C- 1203 
ARPA  Order  No.  3119 

APPROVED  FOR  PUBLIC  RELEASE 
distribution  unlimited 


DISTRIBUTION  STATEMENT  A 

Approved  fat  public  ieleaMC 
Distribution  Unlimited 


USCIPI  Report  690 


ESTIMATION  AND  DETECTION  OF  IMAGES 
DEGRADED  BY  FILM-GRAIN  NOISE 


by 

Firouz  Naderi 


September  1976 


Image  Processing  Institute 
University  of  Southern  California 
University  Park 
Los  Angeles,  California  90007 


This  research  was  supported  by  the  Advanced  Research  Project 
Agency  of  the  Department  of  Defense  and  was  monitored  by  the 
Air  Force  Avionics  Laboratory  under  Contract  No,  F-33615-76-C- 1203, 
ARPA  Order  No.  3119. 


The  views  and  conclusions  in  this  document  are  those  of  the  author 
and  should  not  be  interpreted  as  necessarily  representing  the 
official  policies,  either  expressed  or  implied,  of  the  Advanced 
Research  Projects  Agency  or  the  U.  S.  Government. 


UNCLASSIFIED 


1  DOCUMENT  CONTROL  DATA  •  R  &  D  j 

1  (Security  (iRu/ficalion  ol  title,  body  o  1  aba  tract  and  indexing  annotation  moat  be  entered  when  the  overall  report  la  elaaalllod)  J 

1.  ORIGINATINC  ACTIVITY  (Co IpoMtA  AVlfcWj 

Image  Processing  Institute 

University  of  Southern  California,  University  Park 
Los  Angelas.  California  90007 

la.  REPORT  SECURITY  CLASSIFICATION 

UNCLASSIFIED 

26.  GROUP 

*.  REPORT  TITLR 

ESTIMATION  AND  DETECTION  OF  IMAGES  DEGRADED  BY  FILM- GRAIN 

NOISE 

Technical  Report,  September  19tt 

«•  AUTHORS)  (Flrat  name,  middle  Initial,  laal  name) 

Firouz  Naderi 

I.  REPORT  OATS 

1  September  1976 

7a.  TOTAL  NO.  OF  PACK  7b.  NO.  OF  RE  PI 

160  35 

F-33615-76-C- 1203 

b.  PROJECT  NO. 

c  ARPA  Order  No.  3119 

d. 

be.  oRioiNATor*  report  numieroi 

USCIPI  Report  690 

#6.  OTHER  REPORT  nO(S)  (Any  other  number*  that  may  be  aaal0ted 
thla  report) 

foBLit, 

Approved  for  release:  distribution  unlimited 

11.  supplement  art  no  res 

> 

12.  SPONSORING  MILITARY  ACTIVITY 

Advanced  Research  Projects  Agency 

1400  Wilson  Boulevard 

Arlington,  Virginia  22209 

11.  abstract 

Film-grain  noise  is  a  term  describing  t 
graphic  emulsion  during  the  process  of  rec 
grain  noise  has  been  recently  considered  u 
nature  of  the  noise  is  still  somewhat  misuz 

\  rWis  onal  rsf  fVii  m  atiirlv  Yiaa  to  use 

he  intrinsic  noise  produced  by  a  pheto- 
:ording  an  image  on  film.  Although  film- 
rithin  the  field  of  image  processing,  the 
ider  stood. 

the  theoretical  and  experimental  results 
aphic  scientists  in  order  to  define  film- 
n  theory.  A  detailed  model  for  the 
isented.  There  are  linear  blurring  effects 
s  model  account  for  various  optical 

egment  of  the  model  ^represents  signal 
includes  a  nonlinear  noise  term.  The 
ting  images  according  to  it  and  com- 
"gets  that  were  actually  recorded  on  film. 

:1  is  a  good  representation  of  the  photo- 

film-grain  noise  is  considered  in  two 
detection  theory.  Under  the  topic  of 
reloped  which  explicitly  allows  for  the 
*  adaptively  alters  its  characteristics 
:s  of  an  image.  This  filter  is  shown  to — 

on  film  characteristics  obtained  by  photogr 
grain  noise  within  the  context  of  estimatio 
overall  photographic  imaging  system  is  pr< 
at  the  initial  and  the  final  segments  of  thi 
and  chemical  degradations.  The  middle  s 
dependence  effects  of  film-grain  noise  and 
accuracy  of  this  model  is  tested  by  Simula 
paring  the  results  to  images  of  similar  tai 
These  comparisons  point  out  that  the  mode 
graphic  imaging  system. 

The  restoration  of  images  degraded  by 
different  contexts  -  estimation  theory  and 
lestimation,  a  discrete  Wiener  filter  is  de> 
signal-dependence  of  the  noise.  The  filtei 
based  on  nonstationary  first  order  statist!: 

Security  Ct. tonification 


t  4. 


KEY  WORDS 


III 

ROUE 


3  have  an  advantage  over  the  conventional  Wiener 
filter,  v- 

In  the  case  of  extremely  low  contrast  images 
digitized  by  a  very  small  aperture,  film-grain 
noise  is  so  severe  that  conventional  statistical 
resotration  techniques  have  little  effect.  For 
use  in  this  situation  a  heuristic  algorithm  is  devek 
oped  which  incorporates  some  of  the  vision  pro¬ 
perties  of  the  human  observer.  Bayesian  de¬ 
tection  theory  is  used  to  justify  the  procedure  and 
to  provide  some  insight  into  its  use.  This  algo¬ 
rithm  also  explicitly  includes  the  signal  depen¬ 
dence  of  the  noise  and  has  the  capability  of  greatly 
outperforming  the  human  observer  in  locating 
objects  corrupted  by  very  severe  noise. 

Experimental  results  for  modeling,  the 
adaptive  estimation  filter  and  the  Bayesian  detec¬ 
tion  algorithm  are  presented. 


$$$:;:  3 


14.  Key  Words:  Film-Grain  Noise,  Image 

Processing,  Wiener  Estimation, 
Bayesian  Detection. 


ACKNOWLEDGEMENT 


«* 


I  feel  deeply  indebted  to  the  Chairman  of  my  Dissertation 
Committee,  Dr.  Alexander  Sawchuk.  His  advice  and  constant 
encouragement  during  the  course  of  this  research  was  truly 
invaluable. 

I  would  also  like  to  thank  Professor  Nasser  Nahi  for  his 
guidance  during  my  entire  graduate  study,  and  Professor  John 
Pierce  for  his  interest  in  this  work. 

I  am  very  grateful  to  Dr.  William  Pratt  under  whose  leadership 
the  Image  Processing  Institute  of  USC  grew  and  gave  me  and  many 
of  my  fellow  graduate  students  a  chance  to  do  research  in  this  field. 

My  thanks  also  goes  to  Dr.  Hans  Zweig  for  his  many  helpful 
suggestions  during  preparation  of  Chapters  3  and  4. 


Accession  ?or 


KTIS  GRAM 
DTIC  TAB 
Unannounced 
Justification- 


it 


□ 

□ 


By _ 

Distribution/ 


Diet 


Availability  Codes 
Avail  and/or 
Special 


A 


iii 


ABSTRACT 


—  ..... 


e  . 


Film-grain  noise  is  a  term  describing  the  intrinsic  noise  * 

produced  by  a  photographic  emulsion  during  the  process  of  recording 
an  image  on  film.  Although  film-grain  noise  has  been  recently 
considered  within  the  field  of  image  processing,  the  nature  of  the 
noise  is  still  somewhat  misunderstood. 

One  goal  of  this  study  has  been  to  use  the  theoretical  and  experi¬ 
mental  results  on  film  characteristics  obtained  by  photographic 
scientists  in  order  to  define  film-grain  noise  within  the  context  of 
estimation  theory.  A  detailed  model  for  the  overall  photographic 
imaging  system  is  presented.  There  are  linear  blurring  effects  at 
the  initial  and  the  final  segments  of  this  model  to  account  for  various 
optical  and  chemical  degradations.  The  middle  segment  of  the  model 
represents  signal  dependence  effects  of  film-grain  noise  and  includes 
a  nonlinear  noise  term.  The  accuracy  of  this  model  is  tested  by 
simulating  images  according  to  it  and  comparing  the  results  to 
images  of  similar  targets  that  were  actually  recorded  on  film. 

These  comparisons  point  out  that  the  model  is  a  good  representation 
of  the  photographic  imaging  system. 

The  restoration  of  images  degraded  by  film-grain  noise  is 

i 

considered  in  two  different  contexts  -  estimation  theory  and 
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detection  theory.  Under  the  topic  of  estimation,  a  discrete  Wiener 
filter  is  developed  which  explicitly  allows  for  the  signal-dependence 
of  the  noise.  The  filter  adaptively  alters  its  characteristics  based 
on  nonstationary  first  order  statistics  of  an  image.  This  filter  is 
shown  to  have  an  advantage  over  the  conventional  Wiener  filter. 

In  the  case  of  extremely  low  contrast  images  digitized  by  a 
very  small  aperture,  film-grain  noise  is  so  severe  that  conventional 
statistical  restoration  techniques  have  little  effect.  For  use  in  this 
situation  a  heuristic  algorithm  is  developed  which  incorporates  some 
of  the  vision  properties  of  the  human  observer.  Bayesian  detection 
theory  is  used  to  justify  the  procedure  and  to  provide  some  insight 
into  its  use.  This  algorithm  also  explicitly  includes  the  signal 
dependence  of  the  noise  and  has  the  capability  of  greatly  outperform¬ 
ing  the  human  observer  in  locating  objects  corrupted  by  very  severe 
noise. 

Experimental  results  for  modeling,  the  adaptive  estimation 
filter  and  the  Bayesian  detection  algorithm  are  presented. 
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Chapter  1 


INTRODUCTION 


Image  restoration  and  enhancement  are  terms  used  to  describe 
methods  of  removing  different  types  of  degradations  in  imaging 
systems.  With  the  advent  and  improvement  of  digital  computers, 
investigators  have  concentrated  for  the  past  decade  on  digital 
techniques  to  deal  with  the  large  quantities  of  pictorial  data  in 
image  processing. 

The  degradations  that  an  imaging  system  imposes  on  a  picture 
are  often  of  two  types.  There  are  degradations  that  can  be  described 
by  smoothing  operations,  including  effects  due  to  the  finite  resolu¬ 
tion  of  sensors  and  various  optical  and  chemical  effects.  There  are 
also  degradations  due  to  measurement  errors,  and  these  are  usually 
referred  to  as  noise.  Film-grain  noise  is  one  such  disturbance. 

The  goal  of  this  study  is  to  unite  some  results  of  photographic 
scientists  in  modeling  film-grain  noise  with  some  recent  develop¬ 
ments  in  the  field  of  digital  image  processing  to  improve  the  quality 
of  images  degraded  by  severe  film-grain  noise.  In  order  to  make 
this  study  useful  to  workers  both  in  the  fields  of  digital  image 
processing  and  photographic  science,  a  summary  of  relevant  past 
results  is  presented  for  each  topic. 
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The  material  presented  here  is  divided  into  three  major  topics: 
modeling,  estimation  and  detection.  The  following  is  an  outline  of 
this  study  and  a  summary  of  the  research  contributions  in  each.  In 
Chapter  1  we  examine  the  problem  of  film-grain  noise  by  considering 
the  physical  structure  of  photographic  film.  In  particular  we  discuss 
various  factors  effective  film- grain  noise  during  the  exposure  and 
development. 

The  mathematical  modeling  of  film-grain  noise  is  the  subject 
of  Chapter  2.  First  we  review  some  simple  models  suggested  by 
different  researchers.  We  then  use  published  experimental  results 
on  film  characteristics  to  investigate  the  functional  form  of  the 
standard  deviation  of  the  noise.  The  trade-off  between  linear  model¬ 
ing  with  signal  dependent  noise  and  nonlinear  modeling  with  signal- 
independent  noise  is  also  discussed  in  this  chapter.  A  subjective 
evaluation  procedure  for  various  models  is  suggested  and  the 
chapter  concludes  with  a  detailed  model  flexible  enough  to  accomo¬ 
date  different  degradations  imposed  by  an  imaging  system. 

Chapter  3  discusses  restoration  of  film-grain  noise  degraded 
images  under  the  topic  of  estimation.  A  discrete  linear  minimum 
mean  square  error  filter  is  developed  which  explicitly  allows  for  the 
signal-dependence  of  the  noise.  This  filter  adaptively  alters  its 
characteristics  based  on  the  non-stationary  first  order  statistics  of 
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the  image.  This  filter  is  shown  to  have  an  advantage  over  the 
conventional  Wiener  filter. 

Chapters  4  and  5  consider  the  detection  of  extremely  low 
contrast  images  which  have  been  almost  completely  obliterated  by 
film-grain  noise.  In  Chapter  4  we  provide  mathematical  justification 
for  an  algorithm  called  "noise  cheating"  previously  used  by  other 
researchers.  In  Chapter  5  we  first  improve  on  "noise  cheating" 
to  make  it  applicable  to  a  broader  class  of  images  and  to  utilize  some 
of  the  signal-dependent  effects  from  our  model.  We  also  develop  a 
more  versatile  Bayesian  detection  scheme  which  yields  superior 
results  and  extends  the  usefulness  of  the  technique. 

1.  1  Structure  of  Photographic  Film 

Photographic  film  is  probably  the  most  commonly  used  device 
for  detecting  or  recording  light.  Let  us  consider  a  cross  section  of 
a  typical  commercial  film  for  use  in  ordinary  cameras.  As  shown 
in  the  schematic  drawing  of  Fig.  1.1-1,  film  consists  of  an  emulsion 
supported  on  a  firm  base.  The  emulsion  is  composed  of  a  clear 
transparent  gelatin  containing  a  suspension  of  silver  halide  particles. 
These  particles,  or  film  grains  as  they  are  called,  are  photo 
sensitive  and  typically  consist  of  a  lattice  of  silver  ions  and  halide 
ions.  Upon  exposure  to  light,  the  state  of  some  of  these  grains  will 
change  in  such  a  manner  as  to  record  the  spatial  distribution  of  the 
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GELATIN 


SILVER  HALIDE  CRYSTAL 


BASE 


Figure  1.1-1,  Schematic  drawing  of  a  cross  section  of  a 
photographic  film. 


exposing  light.  The  process  of  image  formation  basically  involves 
two  separate  steps: 


1.  Formation  of  Latent  Image.  When  exposed  to  light,  photons 
of  sufficient  energy  will  be  absorbed  by  the  silver  halide,  thus  raising 
an  electron  from  the  valence  band  to  the  conduction  band.  Assuming 
the  halide  crystal  was  initially  electrically  neutral,  the  elevation  of 
the  electron  to  the  conduction  band  must  simultaneously  produce  a 
positively  charge  entity- -a  positive  hole.  At  this  point  one  of  two 
events  may  occur.  Either  the  free  and  mobile  electron  may 
recombine  with  a  positive  hole  to  reproduce  a  neutral  crystal  plus 
some  released  energy  or  the  electron  may  combine  with  a  silver  ion 
to  form  a  quasi-stable  silver  atom.  The  occurrence  of  these  two 
events  is  probabilistic  and  as  we  shall  see  later  this  randomness  is 
indeed  one  of  the  causes  of  film-grain  noise. 

If  a  sufficient  number  of  photons  are  absorbed  by  a  grain  and 
they  in  turn  result  in  silver  atoms,  these  atoms  form  tiny  silver 
specks  called  development  centers  on  the  surface  of  the  exposed  grain 
as  shown  in  Fig.  1.  1-2.  The  size  of  these  specks  are  larger  on  the 
grains  that  were  heavily  exposed  and  much  smaller  on  the  grains  that 
did  not  absorb  enough  photons.  Naturally  there  are  crystals  that  are 
completely  left  unchanged  because  they  were  not  exposed  or  were 
exposed  very  little.  Photographic  scientists  use  the  term  latent 
image  to  describe  the  film  at  this  point.  Although  no  visible  image 
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(a) 

Low  Intensity 


(b) 

High  Intensity 


Figure  1.  1-2.  Electron  micrograph  for  a  uniform  exposure  showing 
development  centers  consisting  of  aggrigates  of 
silver  atoms  on  silver  bromide  emulsion  grains.  [2l 
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has  yet  been  formed,  the  state  of  some  of  the  halide  grains  has  been 
changed  as  to  retain  the  spatial  distribution  of  the  exposing  light. 
These  grains  with  development  centers  will  render  a  visible  image  in 
the  second  step  of  image  formation  process. 

2.  Development  of  Latent  Image.  To  produce  a  visible  image, 
the  exposed  film  is  immersed  in  a  developer  solution.  The  developer 
acts  as  a  reducing  agent  and  completely  changes  the  halide  crystals 
with  stable  development  centers  to  opaque  metallic  silver  grains. 

The  rate  or  the  probability  that  they  are  converted,  however,  depends 
on  the  size  and  the  number  of  these  silver  specks.  It  is  believed 
that  if  three  to  six  silver  atoms  or  more  are  formed  during  the 
exposure,  then  a  stable  development  center  is  formed.  Undeveloped 
crystals  are  washed  away  into  the  developer  solution  leaving  clear 
transparent  gelatin  in  their  place. 

The  question  that  arises  naturally  is  why  photographs  appear 
continuous  when  the  congregations  of  metallic  silver  grains  are 
scattered  randomly  within  a  clear  gelatin.  The  answer  clearly  is  that 
photographs  do  indeed  appear  as  random  distribution  of  opaque  grains 
within  a  transparent  gelatin  if  they  are  viewed  under  sufficient  magni¬ 
fications  as  shown  in  Fig.  1.  1-3.  However  since  the  human  visual 
system  has  a  limited  resolving  capability  and  film  grains  are  small 
in  size,  (see  Table  1.1-1),  regions  in  the  film  consisting  of  a 
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Continuous  tone  positive  proof 


A  16  x  enlargement 


A  100  x  enlargement 


A  series  of  pictures  demonstrating  the  grainy 
appearance  of  photographs  under  sufficient 
enlargement  [l]. 


considerable  number  of  opaque  metallic  silver  grains  appear  to  be  a 
homogeneous  shade  of  black.  On  the  other  hand,  areas  where  there 
are  no  grains  or  where  the  grains  are  very  sparse  appear  to  be 
transparent. 

As  discussed  earlier  in  this  section,  film  is  a  device  for  record¬ 
ing  light.  Indeed,  when  one  looks  at  a  developed  photograph,  one 
gets  a  subjective  feeling  for  the  spatial  distribution  of  the  exposing 
light.  Black  regions  on  the  photograph,  for  example,  indicate  a  high 
intensity  of  the  exposing  light.  In  order  to  establish  some  quantitative 
measure  of  the  exposing  light  or  some  parameter  such  as  its  intensity 
let  us  view  the  film  as  a  system  and  define  an  input,  output  and  a 
transfer  function. 


Average 

Average 

Plate  or  Film 

Diam.  (|Jm.  ) 

Area  (|amr 

High-resolution  film 

0.  048 

0.  00188 

Motion-picture -positive 

film 

.  30 

.07 

Positive-type  film 

.  63 

.  31 

Fine-grain  roll  film 

.79 

.49 

Portrait  film 

.  88 

.61 

High-speed  roll  film 

1.  09 

.93 

X-ray  film 

1.71 

2.  30 

Table  1.1-1.  Grain  size  for  various  type  of  films.  [2] 

1 , 2  Film  Viewed  as  a  System 

The  input  to  the  film  is  the  incident  light,  or  more  generally,  the 
exposure  E  defined  as 
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where  I  is  the  intensity  of  the  incident  light  and  t  is  the  exposure  time. 
The  most  common  quantitative  measures  of  the  output  of  a  photo¬ 
graphic  film  are  either  optical  density  or  transmittance.  For  a 
region  R  in  a  developed  film,  transmittance  is  measured  in  the 
following  manner:  the  region  is  illuminated  with  a  light  of  constant 
intensity  I.  and  then  the  intensity  of  the  light  transmitted  to  the  other 
side  of  the  film  through  this  region  is  measured.  Denoting  this 
intensity  by  1^,  the  transmittance  value  for  this  region  is  defined  as 


and  optical  density  is  defined  as 

I. 

D  =  -log1()  T  =  loglQ  •  (1.2-3) 

When  we  look  at  a  photograph,  we  perceive  a  spatial  variation 
in  incident  light  intensity.  Photographic  film  converts  this 
variation  by  appropriate  exposure  and  development  to  spatial  vari¬ 
ations  in  optical  density  or  transmittance  of  the  film.  From  eqs. 
(1.2-2)  and  (1.2-3)  we  note  that  since  I  ^  1^,  optical  density  is  a 
nonnegative  quantity  and  transmittance  T  varies  between  0.0  and 
1.0.  As  we  shall  see  in  Chapter  2,  the  measured  optical  density  for  a 
region  is  directly  proportional  to  the  mass  of  metallic  silver  deposited 
in  that  region  during  the  process  of  exposure  and  development. 


Suppose  the  incident  light  exposing  a  particular  region  R  has  a  high 
intensity.  Most  of  the  halide  grains  within  this  region  will  be 
transformed  to  metallic  silver  grains.  Now,  since  silver  grains  are 
opaque,  in  the  process  of  measuring  the  optical  density  for  this 
region  the  transmitted  intensity  Ifc  of  eq.  (1.2-3)  will  be  much  less 
than  intensity  I.,  Therefore  as  shown  by  eq.  (1.2-3),  a  large  value 
for  optical  density  will  be  recorded.  Thus  high  optical  density  for  a 
region  corresponds  to  a  large  mass  of  silver  in  that  region  which 
in  turn  is  indicative  of  high  intensity  of  the  exposing  light. 

So  far  in  this  section  we  have  established  quantitative  input  and 
output  for  a  film.  What  remains  to  be  discussed  is  a  transfer  function 
linking  the  input  and  the  output  together.  Around  1900,  two  photo¬ 
graphic  scientists,  Hurter  and  Driffield,  performed  a  series  of 
experiments.  They  discovered  that  in  macroscoph  regions,  the 
logarithm  of  the  exposure  E  (input  to  the  photographic  system)  and 
the  optical  density  (output  of  the  photographic  system)  are  related 
through  a  function  which  has  become  known  as  the  Hurter  and 
Driffield  (H  and  D)  curve  or  D-log  E  characteristic  curve.  A  typical 
H  and  D  curve  is  shown  in  Fig.  1,2-1. 

If  the  film  is  operated  in  the  monotonic  region  of  this  curve,  a 
measured  optical  density  would  uniquely  determine  the  input  exposure, 
at  least  in  the  macroscopic  regions  for  which  this  curve  is  obtained. 


Thus  by  measuring  optical  density  in  various  regions  of  a  developed 
film  one  can  determine  the  spatial  distribution  of  the  exposing  light. 

However,  if  we  measure  the  optical  density  of  a  developed  film 
not  in  macroscopic  regions  but  in  microscopic  regions,  we  note  that 
the  measured  optical  density  does  not  relate  to  the  input  exposure  in  a 
straightforward  manner  and  film  behaves  more  like  a  stochastic 
system.  This  is  what  we  will  examine  in  the  next  section. 

1 .  3  Film-Grain  Noise 

Suppose  we  expose  a  film  with  light  having  absolutely  no  spatial 
variation,  or  in  other  words,  perfectly  constant  intensity.  If  this 
film  is  viewed  with  a  naked  eye  after  careful  development  we  see  that 
a  constant  shade  of  gray  has  been  registered  across  the  film.  So  far 
everything  is  according  to  expectations;  an  input  with  a  constant  level 
has  resulted  in  a  subjectively  constant  output.  If  we  measure  optical 
density  in  various  macroscopic  regions,  perhaps  two  or  three 
millimeters  in  diameter,  as  predicted  by  the  H  and  D  curve,  the 
same  value  for  the  optical  density  is  measured  in  the  various  regions. 

However,  if  we  use  a  microdensitometer  with  a  small  aperture 
of  only  a  few  microns  in  diameter  to  measure  optical  density  for 
several  microscopic  regions  in  the  film,  we  note  that  different 
readings  are  obtained  for  different  regions  even  though  the  exposing 
light  for  all  of  the  regions  was  the  same.  Considering  photographic 
film  as  a  space -invariant  system,  these  fluctuations  in  the  output 
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from  one  region  to  another,  when  in  fact  the  input  in  all  of  the  regions 
is  the  same,  indicates  that  the  system  is  not  noise  free.  There  are 
several  reasons  for  this  phenomenon,  some  of  which  are  discussed 
below; 

1.  As  discussed  in  section  1.1,  photographic  film  is  composed 
of  microscopic  silver  grains  and  thus  is  inherently  a  discrete 
medium.  It  should  be  expected,  therefore,  that  under  sufficient 
magnification  it  would  appear  inhomogeneous.  In  fact,  if  transmit¬ 
tance  T  defined  in  eq.  (1.2-2)  could  be  measured  at  any  point  (x,y) 
it  would  be  either  zero  or  one  depending  on  whether  the  point  was 
covered  by  an  opaque  silver  grain(s)  or  not.  Therefore  film  can  be 
taken  to  be  a  binary  medium  on  a  microscopic  scale. 

2.  Even  though  the  incident  light  may  be  spatially  uniform  over 
the  extent  of  the  film,  from  a  quantum  viewpoint  the  number  of 
photons  arriving  at  the  surface  of  the  film  per  unit  time  is  not  con¬ 
stant  over  the  entire  surface  but  has  a  poisson  distrubution.  Thus, 
each  silver  halide  grain  in  the  film  is  struck  with  a  different 
number  of  photons  and  therefore  some  have  more  chance  to  reduce  to 
metallic  silver  during  the  development.  This  effect  is  particularly 
significant  when  the  incident  light  is  of  a  very  low  intensity. 

3.  As  discussed  in  section  1.1,  both  the  formation  of  the  latent 
image  and  its  development  are  probabilistic  in  nature.  Two  seem¬ 
ingly  identical  silver  halide  grains  struck  with  the  same  number  of 


photons  react  differently.  One  might  eventually  be  converted  to  a 
silver  grain  whereas  the  other  might  not. 

4.  It  has  been  established  that  silver  grains  for  a  uniformly 
exposed  and  developed  film  are  randomly  scattered  throughout  the 
film  [3].  Their  distribution  is  nearly  binomial  and  can  be  approxi¬ 
mated  by  poisson  if  the  grains  are  assumed  indefinitely  small.  If 

a  microdensitometer  is  used  to  scan  the  film,  the  number  of  grains 
that  fall  within  the  area  covered  by  the  scanning  aperture  is  different 
at  different  locations.  Since  measured  optical  density  for  a  region 
is  directly  proportional  to  the  mass  of  metallic  silver  deposited  in 
that  region,  the  randomness  associated  with  the  number  of  grains  that 
fall  within  various  regions  contributes  to  randomness  in  measured 
optical  density. 

5.  It  is  practically  impossible  to  make  an  emulsion  with 
constant  size  grains.  Thus  the  variation  in  grain  size  will  also 
affect  measured  optical  density  due  to  relationship  between  optical 
density  and  the  mass  of  silver. 

In  summary,  the  fluctuations  in  the  number,  size  and  configura¬ 
tion  of  grains  from  one  small  subregion  to  the  next  causes  corres¬ 
ponding  fluctuations  in  the  optical  density  (or  transmittance)  of 
microscopic  regions  in  a  uniformly  exposed  and  developed  film,  and 
this  produces  the  noise  in  the  photographic  system. 
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When  this  noise  is  measured  objectively  by  various  types  of 
instrumentation,  it  is  called  granularity,  whereas  the  subjective 
impression  of  film  noise  in  a  photographic  enlargement  is  call' d 
graininess. 


References 


tl"l  J.  F.  Hamilton,  W.  H.  Lawton  and  E.  A.  Trabka,  "Some 

Spatial  and  Temporal  Point  Processing  in  Photographic  Science,  " 
Invited  paper  at  the  International  Conference  on  Stochastic  Point 
Processes:  Statistical  Analysis,  Theory  and  Application,  IBM 
Research  Center,  Yorktown  Heights,  NY,  August  2-7,  1971. 

[2"|  W.  Thomas,  SPSE  Handbook  of  Photographic  Science  and 
Engineering,  John  Wiley  &  Sons,  NY,  p,  768,  1973. 

[3]  C.  E.  K.  Mees  and  T.  H.  James,  The  Theory  of  the  Photo¬ 
graphic  Process,  3rd  Edition,  Macmillan,  NY,  1966,  p.  525- 
527. 


16 


t 


Chapter  2 


MODELING 

One  of  the  aims  of  science  is  to  understand  the  nature  of  the  real 
world  based  on  simplified  artificial  models.  The  laws  and  relation¬ 
ships  that  evolve  from  these  idealized  models  are  approximations. 

In  the  case  of  most  real  world  processes,  these  idealized  models 
must  be  altered  or  supplemented  to  reflect  the  complex  nature  of  the 
process.  The  tradeoff  is  usually  between  the  utility  of  the  simplified 
models  and  the  accuracy  of  the  more  complex  ones. 

As  briefly  reviewed  in  the  first  chapter,  the  formation  of  an 
image  on  photographic  film  is  a  highly  complex  optical  and  chemical 
process.  Modeling  this  process  with  a  high  degree  of  accuracy,  if 
at  all  possible,  often  results  in  models  that  are  too  complex  and 
unsuitable  for  use  in  any  subsequent  mathematical  processing.  On 
the  other  hand,  the  reliability  of  restoration  techniques  such  as 
estimation  or  detection  is  directly  related  to  the  degree  by  which  the 
underlying  mathematical  model  simulates  the  actual  physical  process. 
Therefore,  oversimplification  of  the  model  makes  the  subsequent 
restoration  technique  very  sub-optimal. 

With  these  thoughts  in  mind,  the  goal  of  this  chapter  is  to 
present  a  detailed  model  for  the  overall  imaging  system  which  is 
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reasonably  accurate  and  yet  mathematically  tractable. 

In  section  2.  1  we  review  some  simplified  models  which  give  us 
a  general  expression  for  the  standard  deviation  of  noise  of  the  photo¬ 
graphic  system.  In  section  2.2  we  see  how  various  quantities,  such 
as  the  standard  deviation,  can  be  used  as  an  objective  measure  of 
graininess  of  films.  Some  experimental  results  are  analyzed  in 
section  2.  3  and  this  analysis  leads  to  a  more  accurate  form  for  the 
standard  deviation.  The  probability  distribution  of  optical  density  is 
the  subject  of  section  2.4.  Using  the  results  of  the  first  four  sections, 
in  section  2.  5  we  obtain  some  models  for  the  optical  density  in  the 

(context  of  estimation  theory.  In  section  2.6  we  discuss  the  tradeoff 
between  various  models.  A  procedure  for  the  evaluation  of  various 
mathematical  models  of  the  photographic  process  is  outlined  in  section 
2.7,  and  a  detailed  model  for  the  imaging  system  is  suggested  in 
section  2.  8. 

2,  1  Standard  Deviation  of  Film-Grain  Noise 

In  1913  Nutting  gave  a  simplified  model  for  a  developed  photo¬ 
graphic  layer,  expression  optical  density  of  the  image  in  terms  of  the 
number  and  size  of  developed  grains  [l  1 .  Suppose  a  film  is  exposed 
to  a  spatially  uniform  light  and  then  developed.  Nutting  assumed  that 
the  developed  emulsion  layer  may  be  divided  into  k  elementary  levels 
each  about  one  grain  thick  as  shown  in  Fig.  2,  1-1.  The  grains  are 
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assumed  randomly  distributed  within  each  level  with  a  negligible 
degree  of  overlap.  For  the  above  model  Nutting  showed  [2]  that  the 
optical  density  D  can  be  written  as 

D  =  .43  22-  (2.1-1) 

where  a  is  the  size  of  the  developed  grains  (which  is  taken  to  be  the 
same  for  all  the  grains),  A  is  the  size  of  the  scanning  aperture  used 
to  measure  the  optical  density  D  and  n  is  the  total  number  of  grains 
that  fall  within  the  aperture  area  at  any  given  point. 

Despite  the  many  simplifying  assumptions  that  were  made  in 
deriving  eq.  (2.  1-1),  Nutting's  model  has  proved  of  lasting  benefit 
and  this  equation  and  variations  of  it  are  still  widely  used. 

As  mentioned  in  chapter  1,  the  grains  in  a  uniformly  exposed  and 
developed  film  are  randomly  distributed  within  the  gelatin.  There¬ 
fore  n  in  eq.  (2.  1-1)  is  a  random  variable.  This  randomness  in  turn 
gives  rise  to  fluctuations  in  the  measured  optical  density  D  of  the 
uniformly  exposed  film.  As  we  said  earlier  it  is  this  fluctuation  in 
the  optical  density  of  a  uniformly  exposed  and  developed  film  that 
accounts  for  the  noisy  nature  of  photographic  film  as  a  measuring 
device  for  light.  The  standard  deviation  of  D  may  be  taken  as  a 
measure  of  this  noise  and  can  be  calculated  from  eq.  (2.1-1).  Letting 
E  be  the  ensemble  average  operator  we  have: 


E(D) 


E(n)«  a 
A 


(2. 1-2) 


and 


.43 


2 

o  .  a 
n 


A 


(2.1-3) 


where  and  are  variances  of  D  and  n  respectively.  The  distribu¬ 
tion  of  n  can  be  modeled  by  a  binomial  distribution  [3*]  or,  if  —  is 

A 

small,  by  a  poisson  distribution.  Since  the  first  two  moments  of  the 

2 

poisson  distribution  are  equal,  we  have  E(n)  =  a  ,  and  using  this  fact 

n 

in  the  above  two  equations  we  get 


a 

D 


(2.1-4) 


where  D  =  E(D).  Equation  (2.1-4)  indicates  that  the  fluctuation  in 
optical  density  is  dependent  on  three  main  factors: 

a)  The  fluctuation  is  inversely  proportional  to  the  scanning 
aperture  used  in  measuring  D;  it  increases  as  the  aperture 
decreases. 

b)  As  seen  in  Table  1.1-1  the  mean  grain  size  varies  for 
various  films.  Equation  (2.  1-4)  points  out  that  the  fluctua¬ 
tion  in  the  optical  density  is  greater  in  high  speed  films 
which  utilize  large  grains  and  is  smaller  in  high  resolution 
films  which  employ  smaller  grains. 

c)  The  variation  in  density  is  dependent  on  the  mean  density  D. 

Thus  if  two  frames  of  a  film  are  exposed  uniformly,  one  with 
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exposure  Ej  and  the  other  with  exposure  E^,  and  these 
frames  are  then  scanned  with  the  same  size  aperture,  the 
fluctuation  in  the  density  of  the  frame  exposed  to  the  higher 
exposure  (which  is  thus  developed  to  a  higher  mean  density) 
is  greater.  Some  typical  values  for  are  shown  in  Table 
2.  1-1. 

As  we  will  see  in  the  next  section,  o  can  be  used  as  a  measure 

D 

of  the  graininess  of  a  film. 

2.  2  Objective  Measures  of  Graininess 

Graininess  is  the  impression  of  inhomogenity  in  a  magnified 
image.  It  has  been  of  primary  importance  to  the  photographic  scien¬ 
tists  to  establish  an  objective  measure  to  rank  various  types  of  films 
according  to  their  subjective  graininess. 

Material  o(D) 

Tri-X  Aerecon  Type  8403  0.048 

Plus-X  Aerecon  Type  8401  0.034 

Panatomic-X  Aerial  Type  3400  (Estar  thin  base)  0.020 

Kodachrome  11  for  daylight  0.012 

Special  high-definition  aerial  film  (gray  base)  Type  SO-243  0.0074 

High-resolution  plate  0.0025 

Obtained  for  recommended  development  and  under  the  following 
scanning  conditions:  Net  density  of  sample  above  support  =1,0; 
circular  scanning-aperture  diameter  =  48  )im. 


Table  2,  1-1.  Standard  deviation  of  optical  density  for  typical 
Kodak  films  and  plates  [4]  . 


Bricout  [5l  was  among  the  first  to  suggest  exposing  various 
types  of  films  to  a  uniform  exposure  and  then  using  o( D),  described 


in  eq.  (2.1-4),  as  an  objective  measure  of  their  graininess. 

Trying  to  establish  a  single  experimental  quantity  to  measure 
the  graininess  of  photographic  materials,  Selwyn  [6"|  in  1935 
published  a  paper  in  which  he  hypothesized  that  the  probability 
P(D,AD)  that  an  area  A,  chosen  at  random  on  a  uniformly  exposed 
film,  would  have  a  density  different  from  the  mean  density  of  the 
film  by  an  amount  in  the  range  of  [D,  D+AD]  ,  is  a  function  of  A,  D, 

AD  and  a  constant  G  which  he  called  the  "graininess"  constant.  He 
theorized  that  P(D,  AD)  is  proportional  to  AD  for  vanishing  by  small 
values  of  AD  so  that 

P(D,AD)  =  f(G,D,A)-AD  (2.2-’) 

where  f  is  a  function  to  be  determined.  Following  this  hypothesis  and 
using  dimensional  theory  he  showed  the  functional  form  of  f  to  be: 


P(D,  AD) 


A  -(D2A/G2)  .n 

^  C  • oD  • 

rrG2 


(2.2-2) 


In  the  derivation  of  the  above  equation  he  assumed  the  size  of  the 
grains  to  be  sufficiently  small  with  respect  to  A  so  that  adjacent 
nonoverlapping  areas  are  independent. 

Equation  (2.2-2)  suggests  that  the  optical  density  has  a  Gaussian 


distribution  with  variance  77  . 

2A 


The  relationship 
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Gd  =  (2A)2.aD  (2.2-3) 

has  become  known  as  Selwyn's  law  and  G  is  called  the  granularity  or 
Selwyn  coefficient.  This  coefficient  was  suggested  as  an  objective 
measure  of  graininess.  In  eq.  (2.2-3),  the  subscript  on  G  is  included 
to  emphasize  that  the  relationship  defined  through  this  equation  makes 
sense  only  for  a  given  mean  density.  As  predicted  by  eq.  (2.  1-4)  a 
change  in  the  mean  density  results  in  a  change  in  which  in  turn 
alters  the  value  of  the  constant  G. 

2.  3  A  More  Correct  Version  of  the  Standard  Deviation 

After  Selwyn  introduced  the  notion  of  granularity  constant, 
many  workers  within  the  next  twenty  years  performed  a  series  of 
experiments  in  connection  with  Selwyn's  law  [5*1,  [7]  ,  [8]  .  Their 
conclusions  were  quite  varied  with  various  experimenters  reporting 
the  exponent  in  the  eq.  (2.2-3)  to  be  anywhere  in  the  range  of  0.265- 
0.5.  However,  due  to  the  unreliability  of  the  measuring  devices, 
particularly  the  microdensitometers,  most  of  the  experimental 
results  of  this  period  are  of  questionable  value. 

Finally,  Higgins  and  Stultz  [9]  ,  armed  with  the  results  of 
previous  researchers  and  improved  measuring  apparatus  set  out  to 
experimentally  examine  Selwyn's  law.  Their  experiment  resulted  in  a 
set  of  data  published  in  1959.  In  this  experiment  they  examined  four 
types  of  films  ranging  from  high  speed  Kodak  Royal-X  to  fine-grained 


24 


Eastman  Fine  Grain  Release  Positive  film. 

Six  samples  of  each  of  the  four  films  were  uniformly  exposed 
and  developed  to  six  different  optical  density  levels.  Then  each  of  the 
samples  were  scanned  with  seven  different  apertures  ranging  in  size 
from  7.25  mm,  to  384  (Jm.in  diameter.  The  standard  deviation  was 
calculated  for  the  data  obtained  with  each  of  the  seven  apertures  and 
values  of  G  were  then  tabulated  using  eq.  (2.2-3). 

The  results  of  Higgins  and  Stultz  experiments  are  summarized 
in  Figs.  2.3-1  to  2.3-4,  In  each  figure,  G  is  plotted  against  the 
square  root  of  the  scanning  aperture  area  for  all  of  the  six  optical 
density  levels.  From  eq.  (2.2-3)  it  is  clear  that  Selwyn's  law  pre¬ 
dicts  that  the  graphs  in  these  figures  should  be  horizontal  lines  for 
each  given  density.  Higgins  and  Stultz  concluded  that  their  data 
verified  Selwyn's  law  and  the  deviation  from  horizontal  lines  were 
attributed  to  measuring  errors  and  the  impracticality  of  producing 
truly  uniformly  exposed  films. 

From  Figs.  2.3-1  to  2,  3-4  it  is  apparent  that  G  is  not  a  constant 
for  a  very  wide  range  of  apertures.  The  failure  appears  to  be 
consistently  pronounced  at  large  aperture  sizes.  Furthermore, 
in  the  case  of  coarse-grained  Royal-X  film,  the  smallest  aperture 
seems  to  be  outside  the  range  foi  which  Selwyn's  law  holds.  This 


may  be  expected  because  Selwyn  assumes  grain  sizes  to  be  much 
smaller  than  the  scanning  aperture  in  the  deviation  of  eq.  (2.2-2). 
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granularity  constant  G  vs.  square  root  of  the  scanning  aperture 
r  Kodak  Royal-X  Pan  Film  [9]  .  Density  levels:  O  1.12;  A  0.94; 


granularity  constant  G  vs.  square  root  of  the  scanning  aperture 
r  Kodak  Pan-X  Film  [9]  .  Density  levels:  O  0.96,  AO.  75,0  0. 


The  data  obtained  by  Higgins  and  Stultz  can  also  be  used  to  verify 
the  |  exponent  of  D  in  eq.  (2.  1  -4),  although  Higgins  and  Stultz  did  not 
do  so  in  their  paper.  To  do  this  we  rewrite  eq.  (2.  1-4)  in  the  more 


general  form 


where 


and 


o 

D 


—  P 

=  k(D) 


kj  =  J  A 
k2  =  y.  43a 


(2.3-1) 

(2. 3-2) 

(2.3-3) 

(2.3-4) 


In  the  future,  throughout  this  study  we  will  refer  to  k  as  the  scanning 
constant.  As  shown  in  equations  (2.3-3)  and  (2.  3-4),  the  value  of  k 
depends  on  the  constant  k^  by  the  aperture  used  for  scanning  and 
depends  on  the  grain  size  constant  k^  by  the  type  of  film  used. 
Combining  equations  (2.2-3)  and  (2.3-1)  we  get 


G  =  JZ  .k2*(D)P  . 


(2.3-5) 


Taking  the  logarithm  of  both  sides  of  the  above  equation  we  have 


An  G  =  P.  An  D  +  C  (2.  3-6) 

where 

C  =  An(/Tk2).  (2.3-7) 
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Equation  (2.3-6)  is  known  as  the  Siedentopf  [lo]  law.  According 
to  eq.  (2,  3-6)  if  several  frames  of  a  given  film  are  uniformly  exposed 
and  developed,  each  to  a  different  density,  and  the  granularity  cons¬ 
tant  G  for  each  frame  is  calculated,  then  the  plot  of  An  G  vs  An  D 
should  follow  a  straight  line  with  slope  P. 

To  experimentally  test  these  results,  we  used  the  Higgins  and 
Stultz  data  to  see  if  the  slope  of  such  a  line  would  be  \  as  predicted 
by  eq.  (2.1-4).  In  Figs.  (2.  3- 1 )  to  (2.  3-4)  at  each  given  density 
seven  values  for  G  are  given  one  for  each  aperture  size.  An  average 
of  these  seven  values  was  obtained  and  the  logarithm  of  these  averages 
were  plotted  against  the  logarithm  of  the  densities.  Using  least 
squares  techniques,  a  straight  line  was  fit  to  the  data.  The  results 
are  shown  by  solid  lines  in  Figs.  2.  3-5  to  2.  3-8.  A  fair  degree  of 
deviation  from  the  straight  line  is  observed  in  Figs.  2.3-7  and  2.  3-8, 
and  this  is  expected  considering  the  deviation  from  horizontal  lines  in 
Figs.  2.  3-3  and  2.  3-4. 

Next  a  second  set  of  lines  were  obtained  by  plotting  the  loga¬ 
rithm  of  the  densities  vs.  the  logarithm  of  the  average  values  for  G 
taken  not  over  the  entire  range  of  the  apertures  used  in  Figs.  2.3-1 
to  2.  3-4  but  only  over  the  three  smallest  apertures.  This  was  done 
because  the  lines  in  Figs.  2.3-1  to  2,3-4  are  nearly  horizontal  over 
the  smallest  three  apertures.  These  new  plots  are  shown  by  broken 
lines  in  Figs,  2,  3-5  to  2,  3-8.  Note  that  the  data  now  follows  a 
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Figure  2.  3-5.  Plot  of  logarithm  of  the  granularity  constant  G  vs.  the  logari 
density  for  Kodak  Royal-X  Pan  film. 


of  the  mean  density  for  Kodak  Panatomic-X  Film. 


straight  line  very  closely  in  each  case.  The  slopes  and  intercept 
points  of  these  lines  corresponding  to  constants  P  and  C,  respectively, 
in  eq.  (2.3-6)  along  with  the  grain  size  constant  were  calculated 
and  are  listed  in  the  above  figures. 

These  plots  demonstrate  that  for  a  reasonable  range  of  apertures, 
the  standard  deviation  is  of  the  general  form  given  in  eq.  (2.  3-1). 
The  value  of  exponent  P  however  appears  to  be  less  than  0.  5,  which 
was  suggested  by  eq.  (2.  1-4)  through  Nutting's  model,  and  is  more 
likely  in  the  range  0.  3-0.4.  Therefore  a  more  reasonably  form  for 

°DiS 

k2  -7T.1  /3 

aD  =  7A  (D)  (2-3‘8) 

where  k^  depends  on  the  type  of  film  and  is  the  range  0.45-2.36  for 
the  films  used  in  the  Higgins  and  Stultz  study. 

Equation  (2.  1-4)  in  which  exponent  of  D  is  shown  to  be  \  was 
derived  based  on  the  assumption  that  the  grains  in  the  developed 
emulsion  all  have  the  same  size.  Haugh  [ll]  allowed  the  grain  size 
to  follow  a  distribution  and  theoretically  showed  that  the  exponent  P 
of  eq.  (2.  3-1 )  is  of  the  form 

P  =  J~y  (2.3-9) 

where  y  is  a  positive  constant.  This  then  is  in  agreement  with  the 
experimental  results  just  discussed  and  another  justification  for 
taking 

36 


t 


(2. 3-10) 


D 


.  k,D,1/3 


as  the  expression  for  the  standard  deviation  rr^. 

2. 4  Distribution  of  the  Optical  Density 

In  previous  sections  of  this  chapter  we  noted  that  a  uniformly 
exposed  and  developed  film  subjectively  appears  to  consist  of  a 
uniform  shade  of  gray.  However,  in  examining  microscopic  regions, 
one  finds  that  the  optical  density  for  these  regions  fluctuates  about 
the  mean  density  of  the  film  and  the  standard  deviation  of  the  fluc¬ 
tuation  has  the  form  given  in  eq.  (2.  3-10).  In  this  section  we  investi¬ 
gate  the  probability  distribution  of  these  fluctuations. 

The  probability  distribution  of  optical  density  in  microscopic 
regions  of  a  uniformly  exposed  and  developed  film  has  been  studied  by 
photographic  scientists  for  a  long  time.  Most  researchers  have 
reported  Gaussian  or  near  Gaussian  distributions.  Some  of  these 
reports  were  made  by  Bricout  [5"],  Van  Kreveld  [l2"]and  Selwyn  [13"]. 
On  the  other  hand,  some  other  investigators  such  as  Debot  £l4"land 
Berwart  [15"]  reported  some  skewness  and  asymmetry  in  their 
distribution  curves.  Unfortunately,  most  of  these  experiments  were 
done  30  to  40  years  ago  when  accurate  instruments  were  not  available. 
We  now  discuss  the  assumption  of  Gaussianity  and  show  that  it  should 
be  accepted  with  some  degree  of  reservation. 

In  chapter  1,  the  optical  density  for  a  region  R  in  a  developed 
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film  was  defined  as 


D 


■lo8ior 


(2.4-1) 


1 

where  I  was  the  intensity  of  the  light  transmitted  through  region  R 
when  illuminated  by  a  light  of  constant  intensity  I..  Theoretically  1^ 
can  be  zero,  corresponding  to  infiri  te  optical  density.  However,  as 
shown  in  characteristic  curve  of  Fig.  1,2-1,  all  films  reach  a 
saturation  point  beyond  which  any  increase  in  exposure  will  not  yield 
higher  density.  In  practice  densities  above  2.5  are  seldom  en¬ 
countered  for  ordinary  photographic  films. 

In  eq.  (2.4-1),  I.  is  always  greater  or  equal  to  I  .  Therefore, 

1  L 

the  optical  density  D  is  a  nonnegative  quantity.  One  can  therefore 
assume  that  observed  values  of  optical  density  are  approximately  in 
the  range  of  0.0-2.  5.  We  now  will  examine  assumptions  for 
Gaussianity  and  standard  deviation  as  given  by  eq.  (2.3-10)  to 
determine  if  they  are  consistent  within  this  density  range.  We  first 
examine  the  upper  end  of  this  range  by  assuming  a  film  to  be 
uniformly  exposed  and  developed  to  a  very  high  density.  Equation 
(2.3-10)  suggests  that  the  standard  deviation  monotonically  increases 
with  the  mean  density.  However,  when  a  film  is  thoroughly  exposed, 
almost  all  the  halide  grains  will  be  transformed  to  silver  grains  and 
the  film  will  develop  to  an  almost  uniformly  opaque  material  with  not 
much  deviation  from  the  mean  density.  Therefore  it  appears  that 


- 


t 
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above  a  certain  density  eq.  (2.3-10)  is  not  valid. 

As  evident  from  eq.  (2.4-1),  measuring  optical  density  by  a 
microdensitometer  or  other  measuring  devices,  requires  measuring 
It»  If  the  film  is  heavily  exposed,  I  will  be  very  small  and  recording 
would  become  exceedingly  difficult  due  to  the  noise  in  the  measur¬ 
ing  device.  For  this  reason  it  has  been  difficult  to  obtain  accurate 
data  for  a  film  developed  to  a  very  high  mean  density. 

Considering  the  lower  end  of  the  density  range  and  keeping  in 
mind  that  density  is  nonnegative,  we  examine  the  following  example. 
Suppose  a  uniformly  exposed  Kodak  Panatomic-X  film  is  scanned  witn 
an  aperture  of  size  1(1  |im.  in  diameter.  According  to  Fig.  2.3-7, 
the  grain  size  constant  for  this  film  is  approximately  1.2.  Thus  the 
standard  deviation  of  the  optical  density  fluctuation  when  this  film  is 
scanned  with  the  above  aperture  is  given  by  eq.  (2.  3-8)  and  is 

aD  =  .  12(D)1  /3.  (2.4-2) 

The  smallest  mean  density  D  about  which  density  fluctuation 
can  be  Gaussian  with  standard  deviation  of  the  form  given  by  eq. 

(2,4-2)  has  to  be  at  least  3o^  above  zero  so  that  negative  values  for 
the  optical  density  have  a  very  small  probability.  From.  eq.  (2.4-2) 
we  see  that  this  density  is  approximately  D  =  0.216,  corresponding 
to  a  standard  deviation  of  o  =  0.072.  Figure  2.4-la  shows  such  a 
distribution.  This  figure  and  eq.  (2.4-2)  point  out  that  the  distribution 
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of  the  optical  density  for  a  uniformly  exposed  Panatomic-X  film 
scanned  with  an  aperture  of  sizelO  >am.  is  either  not  Gaussian  or 
does  not  have  standard  deviation  of  the  form  given  by  eq.  (2.4-2), 
when  the  mean  density  D  is  in  the  range  [0.0-0.216*]  . 

This  range  will  be  even  wider  if  a  smaller  aperture  is  used  for 
the  scanning.  If,  for  example,  the  diameter  of  the  scanning  aperture 
is  only  5  |jm. ,  then  as  shown  in  Fig.  2.4-lb,  the  smallest  mean 
density  for  which  D-3a^  is  nonnegative  is  0.63.  Using  the  same 
argument  as  above,  for  this  size  aperture  the  density  distribution  will 
be  non-Gaussian  if  the  mean  density  is  in  the  range  [0.0,0.63]  . 

If  the  aperture  size  is  decreased  still  further  then  the  ratio 
a/A  of  the  grain  size  to  aperture  size  will  not  be  small  enough  for 
either  eq.  (2.2-2)  or  eq.  (2,  3-8)  to  hold  and  the  density  distribution 
will  not  be  Gaussian  in  any  range. 

Various  researchers  [14]  - [l  5 have  reported  that  the  density 
distribution  at  very  low  levels  is  skewed.  Thus  one  might  model  the 
distribution  as  a  Rayleigh  distribution  up  to  some  point  and  by  a 
Gaussian  distribution  beyond  this  point.  The  point  would  depend  on 
the  type  of  film  and  the  scanning  aperture  size. 

If  this  choice  of  modeling  appears  unattractive  because  two 
different  distributions  are  involved,  then  one  might  consider  modeling 
by  a  Rician  distribution.  The  Rician  distribution  resembles  a 
Rayleigh  distribution  for  a  mean  close  to  zero  and  approaches  a 


Gaussian  distribution  as  the  mean  increases  [161  . 


In  the  early  days  of  granularity  studies  only  50-100  points  were 
used  for  calculating  the  sample  standard  deviation  as  well  as  density 
distribution  due  to  lack  of  digital  computers  and  accurate  measuring 
devices.  Later  on,  as  many  as  1000  points  were  considered 
although  this  still  might  not  have  been  sufficient  to  obtain  accurate 
results. 

Since  those  early  days,  however,  the  accuracy  of  measuring 
devices  has  been  considerably  improved.  Due  to  this  fact,  along 
with  the  availability  of  digital  computers,  quite  a  bit  can  be  gained  by 
a  more  thorough  study  of  the  distribution  of  optical  density.  This 
task,  however,  is  not  undertaken  in  this  study.  In  most  cases  we 
assume  the  distribution  to  be  Gaussian. 

2.  5  A  Model  for  Optical  Density  in  Context  of  Estimation  Theory 

As  noted  before,  many  of  the  earlier  studies  on  photographic 
film  were  performed  about  1900.  Most  of  these  studies  were  done  by 
photographic  scientists  whose  primary  goal  was  analyzing  different 
films  and  establishing  a  quantitative  measure  ranking  different 
emulsions  according  to  their  subjective  grainy  appearance. 

Research  in  the  area  of  film-grain  noise,  its  nature  and 
modeling,  reached  its  height  during  the  late  forties  to  mid-sixties. 
During  this  period  photographic  scientists  left  a  vast  volume  of 


publications  often  resulting  in  contradictory  findings.  Unfortunately, 
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in  many  of  these  papers  the  mathematical  notions  of  linearity  and 
signal  dependence  are  used  either  too  loosely  or  interchangeably. 

One  goal  of  this  study  is  to  use  the  theoretical  and  experimental 
results  on  film  characteristics  obtained  by  photographic  scientists  in 
order  to  define  film-grain  noise  within  the  contexts  of  estimation 
theory.  It  is  hoped  that  these  models  will  provide  a  stepping  stone 
for  the  eventual  restoration  of  the  degraded  image. 

Earlier  in  this  chapter  we  noted  that  film  can  be  viewed  as  a 
system  in  which  exposure  is  the  input,  optical  density  is  an  output 
and  the  characteristic  function  of  the  film  is  the  transfer  function 
relating  the  input  and  the  output  in  macroscopic  regions.  Thus  the 
optical  density  in  macroscopic  regions  of  a  uniformly  exposed  film 
can  be  written  as 

D  =  g(E)  (2.5-1) 

s 

where  E  is  a  spatially  uniform  input  exposure,  g  is  the  characteristic 
function  of  the  film  and  Dg  is  the  optical  density  registered  on  the 
film. 

As  described  in  section  1.3,  the  optical  density  of  a  uniformly 

exposed  film  examined  in  very  small  regions  of  only  a  few  microns 

in  diameter  fluctuates  about  D  ,  with  the  fluctuation  being  near 

s 

Gaussian  with  standard  deviation  of  the  form  given  in  eq.  (2.  3-10). 


Thus  we  can  write 
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where  N(0,  1)  is  the  zero  mean,  unit  variance  Gaussian  density.  In 

the  above  equation  DQis  the  density  registered  in  microscopic 

regions  on  film  and  n  is  the  noise  of  the  photographic  film,  taken  to 

be  a  zero-mean  Gaussian  random  variable  with  standard  deviation  of 
—  1/3 

k(D  )  .  Equation  (2,5-2)  implies  that  D  is  Gaussian  distributed 

s  ° 

—  —  / 1/3 

with  mean  D  and  standard  deviation  k(D  )  and  statistically 
s  s 

matches  the  measured  characteristics  of  a  uniformly  exposed  film  in 
the  first  two  moments. 

While  uniformly  exposed  films  are  very  helpful  in  studying  the 
statistics  of  film-grain  noise,  most  images  of  interest  are  consider¬ 
ably  different.  We  now  consider  exposures  with  spatial  variations 
and  see  if  the  model  of  eq.  (2.  5-2)  also  describes  the  optical  density 
of  nonuniformly  exposed  films. 

Let  E(x,  y)  be  a  deterministic  function  representing  the  spatial 

variation  of  the  exposure  on  an  imaginary  plane  located  just  in  front 

of  the  film  to  be  exposed.  Dividing  the  surface  of  this  plane  into 
2 

N  cells  of  size  a  Xa  (see  Fig.  2.  5-1)  and  assuming  that  the  spatial 
variation  of  exposure  is  small  within  each  cell,  we  denote  by  E_  the 
exposure  level  within  the  cell  (i,  j). 


Cell  (i  j ) 


Suppose  M  frames  of  a  photographic  film  of  a  particular  type 

are  exposed  with  the  same  exposure  E(x,y)  and  after  development 

2 

their  surfaces  are  divided  into  N  cells  of  size  a  Xa.  We  denote  by 

D™  the  optical  density  registered  in  the  (i,  j)  cell  of  the  m*^  frame. 

Now,  if  photographic  film  were  noise  free,  then  D™  would  have  Lo  be 

identical  for  m  =  1,  ....  M  because  the  exposure  for  all  the  frames  is 

the  same.  Denoting  this  common  value  by  D  ,  the  input  exposure  E.. 

»J  D 

and  D. ,  are  related  through  the  characteristic  curve  by 

ij 


D..  =  g(E. . )  . 

U 


(2.5-3) 


As  we  have  seen,  however,  the  film  noise  causes  D™  to  be  different 
for  each  frame.  Since  the  input  exposure  was  the  same  for  all  of  the 
frames,  then  the  M  density  D. .,  m  =  1,  .  .  .  ,  M  readings  can  be  taken 
as  though  they  were  M  readings  from  a  single  frame  uniformly  ex¬ 
posed  with  exposure  E  .. 

Letting  D. .  ,  m  =  1,  . . . ,  M  be  samples  from  a  random  variable  D 

then  D. .  will  be  Gaussian  distributed  with  mean  D  .  and  standard 
D  ij 

deviation  k(D„)  ^ ,  where  k  is  a  constant  depending  on  the  type  of  the 

film  and  the  size  of  the  cells.  Thus  a  model  reflecting  the  first  two 

moments  of  D. .  would  have  the  form 
ij 


D. .  =  D. .  f  n. . 

U  D  tj 

n..cN(0,k2(D..)2/3)  . 
ij  U 


(2.5-4) 
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In  the  above  equation  D. .  is  nonrandom  and  n..  represent  the  random - 
ness  (noise)  associated  with  photographic  system. 

In  discussions  leading  to  eq.  (2.5-4)  we  assumed  the  input 
exposure  E(x,  y)  to  be  a  deterministic  function.  The  only  randomness 
was  due  to  the  noise  n  introduced  by  the  film.  However  in  most  prac 
tical  cases  E(x,  y)  is  also  random,  so  hat  a  general  model  for 
the  photographic  film  would  have  the  form 

Dg  =  g(E) 

D  =  D  +  n 
o  s 

2  2/3 

neN(0,k  (D  )  ) 

s 

where  in  the  subscripts  (i,  j)  are  dropped  but  understood.  Here  E  is 

the  input  exposure,  D  is  the  resulting  density  in  .he  absence  of 

s 

filrrvgrain  noise,  n  is  the  noise  and  D  is  the  observed  density.  The 

o 

quantities  E,  D  ,  n  and  D  are  all  random  variables, 
s  o 

2.  6  Linearity  vs.  Signal  Dependence 

There  is  a  fair  amount  of  confusion  in  the  literature  regarding  the 
so-called  multiplicative  nature  of  film-grain  noise.  From  eq.  (2.5-6) 
it  is  clear  that  film-grain  noise  can  be  modeled  as  additive  noise. 
However,  this  noise  is  signal  dependent  by  virtue  of  its  variance. 
Alternatively,  the  additive  nature  of  the  noise  can  be  sacrificed  to 
eliminate  the  signal  dependence.  In  this  case  optical  density  can  be 
modeled  as 


(2.5-5) 

(2.5-6) 
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D  =  D  +  k(D  )  .  n 

OSS 

neN(0,  1 ) 

where  the  noise  n  is  Gaussian  wich  zero  mean  and  unit  variance  and  is 
statistically  independent  of  the  signal.  Equation  (2.6-1)  now  repre¬ 
sents  a  nonlinear  model.  Note  that  the  model  of  eq.  (2.5-6)  and  thac  of 
eq.  (2.6-  1)  are  identical  in  the  sense  that  the  random  variable 

eiven  D  has  the  same  first  two  moments  in  either  model.  The 
®  s 

difference  is  that  in  one  model  we  have  additive  signal  dependent 
noise  whereas  in  the  other  model  we  have  nonlinear  observation  with 
signal  independent  noise. 

In  many  estimation  literature  the  term  multiplicative  noise  is 
strictly  used  to  describe  observations  of  the  form 

y  =  s  •  n  (2.6-2) 


where  y  is  the  observation,  s  is  the  signal  and  n  is  the  noise.  For 
this  reason  we  will  not  refer  to  film-grain  noise  of  eq.  (2.  6-1)  as 
multiplicative  noise  lest  that  be  misleading. 

The  model  of  eq.  (2.  5-6)  might  appear  easier  to  work  with 
because  of  the  additivity  of  the  noise.  However  any  advantage  gained 
due  to  linearity  of  the  model  is  more  than  offset  by  the  disadvantage 
of  working  with  signal-dependent  noise.  Thus  in  subsequent  chapters 
we  shall  use  eq.  (2.6-1).  This  model  was  also  suggested  by  Huang  [17] 
to  model  the  random  part  of  film-grain  effects.  Figure  2.6-1  shows 
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the  block  diagram  for  this  model. 

2.  7  Evaluation  of  the  Model 

The  aim  of  this  chapter  has  been  the  development  of  a  model  for 
the  image  forming  and  recording  system  which  can  be  used  in 
restoring  images  degraded  by  film -grain  noise.  The  reliability  of 
any  restoration  scheme  is  directly  related  to  the  accuracy  by  which 
the  underlying  mathematical  model  describes  the  actual  physical 
process. 

Unfortunately  there  does  not  exist  any  well-defined  or  tractable 
fidelity  criterion  by  which  various  models  can  be  evaluated  or 
compared.  For  this  reason,  we  decided  to  test  different  models 
according  to  the  following  procedure.  A  target  was  first  photo¬ 
graphed  and  the  film  developed.  Then  a  section  of  the  film  was 
digitized  using  a  small  aperture.  Next  we  simulated  the  same  section 
of  the  target  in  the  computer  according  to  the  model  which  was  to  be 
evaluated.  Since  any  imaging  model  must  simulate  the  physical 
process,  we  subjectively  compared  the  photographed  and  simulated 
images  to  determine  the  accuracy  of  the  model.  There  is  no 
mathematical  optimality  associated  with  this  type  of  testing  although 
it  is  a  reasonable  procedure,  especially  for  image  processing 
involving  the  human  observer. 

Figure  2.  7- la  shows  the  target  used  for  the  evaluation.  It 
consists  of  two  figure  sevens  cut  out  of  white  and  red  cardboard 
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The  target  used  for  evaluation  of  the  models 


which  are  then  pasted  on  a  black  cardboard.  Photographing  this 

target  on  a  monochrome  film  produces  an  image  with  three  levels. 

This  target  was  then  attached  to  the  side  of  a  car  as  shown  in  Fig. 

2.  7 -lb,  and  photographed  at  various  distances  using  two  different 

types  of  film.  One  was  the  high  speed  Kodak  Tri-X  film  and  the 

other  was  Kodak  Panatomic-X,  a  film  with  much  slower  speed. 

Figure  2.  7-lc  shows  one  of  the  photographs  shot  at  a  long  distance 

on  a  Pan-X  film.  Note  that  the  number  77  is  practically  impossible 

to  see  in  this  picture  because  the  target  of  Fig.  2.7-la  is  mapped 

into  a  square  region  of  dimensions  about  0.5  millimeter  on  each  side 

on  the  negative  of  the  photograph  shown  in  Fig.  2.  7-lc.  This  area 

is  much  too  small  for  the  eye  to  resolve.  Next,  the  negative  of  the 

photograph  shown  in  Fig.  2.7-  lc  and  a  negative  of  exactly  the  same 

scene  recorded  on  a  Tri - X  film  were  digitized  using  various  small 

apertures.  In  each  case  the  digitized  area  was  centered  around  the 

number  77.  Figure  2.7-2  shows  three  such  digitized  images,  all 

consisting  of  256  x  256  pixels.  Figure  2.7-2a  is  the  Pan-X  film 

2 

digitized  with  a  5  x  5  (jam.  )  aperture,  and  Fig.  2.  7-  2b  is  the  same 

2 

film  digitized  with  an  aperture  of  size  2x2  (nm. )  .  As  predicted 

by  the  denominator  of  eq.  (2.3-8),  the  image  digitized  with  the 

smaller  aperture  appears  to  be  more  noisy.  Finally,  Fig.  2.7-3c 

2 

shows  the  Tri-X  film  digitized  with  a  2  x2  (ym.  )  aperture.  Once 

again  comparing  figures  (b)  andfc)  we  see  that  Fig.  (c)  is  more  noisy 
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(b)  Panatomic-X  Film  digitized 
with  a  2  x  2  (am.  )  aperture. 


(a)  Panatomic-X  Filjn  digitized 
with  a  5x5  (um.  )  aperture. 


^  *-  ■  ’■*  /  A  Lit*. 


(c)  Tri-X  Film  digitized  with 
a  2  x  2  (um.  )  aperture. 


Figure  2.  7-2.  Digitized  images  obtained  by  scanning  two  types  of 
film  with  two  different  size  apertures. 


(a)  Actual  image 


Comparison  of  the  actual  image  with  the  image 
simulated  according  to  the  model  of  Fig.  2.6-1 


as  predicted  by  eq.  (2.3-4).  This  results  because  the  average 
grain  size  a  is  larger  for  Tri-X  film  than  for  Pan-X  film. 

We  selected  Fig.  2.7-2b  as  one  of  the  test  targets  to  be  used 
in  evaluation  of  the  model  shown  in  Fig.  2,6-1.  The  two  images  used 
for  the  evaluation  are  shown  in  Fig.  2.7-3.  Figure  2.7-3a  is  the 
actual  photographed  image  and  is  the  same  image  shown  in  Fig. 
2.7-2b.  The  image  simulated  in  the  computer  according  to  the  model 
of  Fig.  2.  6-1  is  shown  in  Fig.  2. 7  - 3b. 

The  question  now  is  whether  the  simulated  image  gives  the 
same  subjective  impression  as  the  actual  photographed  image.  The 
answer  is  clearly  no.  The  edges  of  the  sevens  in  the  simulated  image 
are  very  sharp  whereas  they  are  quite  blurred  in  the  actual  image. 

The  noise  in  Fig.  2.7-3b  has  the  salt  and  pepper  appearance  whereas 
in  Fig.  2.  7-3a  this  is  not  the  case.  This  comparison  points  out  that 
the  model  of  Fig.  2,6-1  alone  is  not  quite  representative  of  the 
imaging  process.  Thus  it  either  needs  to  be  altered  or  augmented, 

2.  8  A  Model  for  Image  Forming  and  Recording  Process 

As  discussed  in  Chapter  1  the  process  of  recording  an  image  on 
a  photographic  film  is  a  rather  complex  process.  Not  all  the  degrada¬ 
tion  is  due  to  film-grain  noise.  There  are  other  complex  optical  and 
chemical  effects  that  contribute  to  the  degradation  of  the  recorded 
image  that  were  neglected  in  the  model  of  Fig.  2,6-1.  Any  accurate 
modeling  should  account  for  these  effects  as  well  as  the  granularity 
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noise. 


Around  I960,  Kelly  [l8]  suggested  a  three  stage  model  which 
included  optical  and  chemical  degradation  effects,  but  ignored  the 
granularity  effects.  Later  Lorber  [19]  proposed  cascading  a  fourth 
stage  to  Kelly’s  model  to  account  for  film- grain  noise. 

Combining  these  results  with  the  discussion  in  the  previous 
seven  sections  of  this  chapter,  a  model  of  the  form  shown  in  Fig. 
2.8-1  is  suggested.  In  this  figure,  I^x,  y)  is  the  intensity  of  the  light 
reflected  from  the  object  and  is  incident  on  the  image  forming  device 
such  as  a  camera.  The  first  of  four  linear  blurs  in  this  model 
represents  atmospheric  degradations  such  as  turbulence  as  well  as 
limiting  effects  of  the  imaging  system  such  as  diffraction  and 
aberrations.  The  point-spread  function  for  this  linear  blur  is  bj(x,y). 
I^x.y),  the  output  of  this  initial  block  is  the  intensity  distribution  of 
the  light  that  actually  strikes  the  film  in  the  camera. 

The  second  linear  blur  models  optical  diffusion  effects  such  as 
scattering  and  halation,  during  the  formation  of  latent  image.  This 
blur  has  a  point-spread  function  b^fx,  y).  The  middle  segment  of  this 
model  is  exactly  the  same  as  the  model  of  Fig.  2.6-1,  and  represents 
the  nonlinear  characteristic  function  g  as  well  as  the  nonlinear  signal- 
independent  film-grain  noise. 

The  third  linear  blur  with  point  spread  function  b^(x,  y)  accounts 
for  degradations  such  as  adjacency  effects  [19]  which  arise  during 
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model  for  the  imaging  system. 


the  development  of  the  film.  Finally,  the  digitization  process  is 
modeled  by  the  fourth  linear  blur.  The  aperture  of  the  scanner  is 
represented  by  point- spread  function  b^(x,  y).  The  output  of  this 
model  is  the  observed  digitized  image  DQ(i,j). 

To  evaluate  this  model  once  again  we  simulated  the  image 
recording  process  according  to  it  and  compared  the  simulated  image 
to  an  actually  photographed  image.  The  results  are  shown  in  Fig. 
2.8-2. 

Figure  2.  8-2a  is  the  actual  image  and  Fig.  2.8-2b  is  the 
simulated  image  according  to  model  of  Fig.  2.6-1  accounting  only 
for  granularity  noise.  Figure  2.  8-2c  is  the  image  simulated 
according  to  the  suggested  model  of  Fig.  2.  8-1. 

Comparing  figures  (a)  and  (c)  it  is  clear  that  the  two  images  are 
very  similar.  Thus  it  is  our  conclusion  that  the  model  suggested  in 
Fig.  2.8-1  is  a  very  reasonable  model  for  describing  the  imaging 
system. 

Figure  2,8-3  describes  the  detailed  procedure  for  simulating 

2.  8-2c.  We  first  started  with  a  simulated  ideal  target  which 

resembled  Fig.  2.7-la  in  size,  shape  and  density.  Next,  this 

256  x  256  image  was  blurred  using  a  Gaussian  blur  of  window  11x11 

pixels  and  standard  deviation  2.0.  Nonlinear  Gaussian  noise  was 

added  to  the  blurred  target  in  accordance  with  the  model  of  Fig. 

2.8-1  with  the  scanning  constant  chosen  to  be  0.3.  Finally,  the  noisy 
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(b)  Simulated  image 

accounting  for  film- grain 
noise  only 


(a)  The  actual  image 


(c)  Simulated  image  accounting 
for  all  degradations 


Comparison  of  the  actual  image  with  images 
simulated  according  to  the  models  of  Fig.  2.6-1 
and  2. 8  -  1 . 


Scanning  constant 


image  was  once  again  blurred  with  a  Gaussian  blur  of  window  size 


7X7  pixel  and  standard  deviation  1. 3  to  obtain  the  final  results. 

It  should  be  noted  that  if  we  wanted  to  simulate  an  image  which 
was  digitized  with  a  larger  aperture,  we  would  have  reduced  the 
window  size  of  the  last  blur  function  in  order  to  get  a  better  resembl¬ 
ance  with  the  actual  image.  This  is  done  because  adjacency  effects 
are  not  as  noticeable  in  images  digitized  with  large  apertures. 

REFERENCES 


[ll  P.  G.  Nutting,  "On  the  Absorption  of  Light  in  Heterogeneous 
Media,  "  Phil.  Mag. ,  Vol.  26,  p.  423,  1913. 

[2*1  J.  C.  Dainty  and  R.  Shaw,  Image  Science,  Academic  Press, 
London,  p.  41,  1974. 

[3]  C.  E.  K.  Mees  and  T.  H.  James,  The  Theory  of  the  Photographic 
Process.  3rd  Ed. ,  Macmillan,  N.  Y.,  1966. 

[4*1  W.  Thomas,  SPSE  Handbook  of  Photographic  Science  and 
Engineering.  John  Wiley  &  Sons,  p.  939,  N.  Y. ,  1973. 

t5]  P.  Bricout,  Compt.  Rend. ,  Vol.  197,  p.  1202,  1933. 

[6"1  E.  W.  H.  Selwyn,  "A  Theory  of  Graininess,  11  The  Phot.  J., 

Vol.  75,  p.  571,  1935. 

t7]  R.  Debot,  Research  (London),  vol.  2,  p.  139,  1949. 

[8]  G.  Hansen  and  P.  H,  Keck,  Z.  Wjss,  Photo. ,  vol.  37,  pp.  86-99, 
1938. 

[9"1  G.  C.  Higgins  and  K.  F.  Stultz,  "Experimental  Study  of  rma 
Granularity  as  a  Function  of  Scanning-Spot  Size,"  J,  Opt.  Soc. 

Am. .  vol.  49,  p.  925,  1959. 

[10]h.  Siedentopf,  "Concerning  Granularity,  Resolution  and  the 

Enlargement  of  Photographic  Negatives,"  Physik  Zeit. .  vol.  38, 
p.  454,  1937. 


61 


[l  1  ]  E.  F.  Haugh,  "A  Structural  Theory  for  Selwyn  Granularity 
Coefficient, 11  The  J.  Phot.  Sci. ,  Vol.  11,  p.  65,  1963. 

[12]  A.  Van  Kreveld,  J.  Opt.  Soc,  Am.  ,  Vol.  26,  p.  170,  1936. 

[13]  E.  W.  H.  Selwyn,  ’’Experimental  on  Nature  of  Graininess," 

The  Phot.  J. ,  p.  513,  Aug.  1939. 

[14]  R.  Debot,  Research  (London).  Vol.  2,  p.  341,  1949. 

[15]  L.  Berwart,  Bull.  Soc.  Roy,  Sci,  Liege,  Vol.  32,  p.  794,  1963. 

[l 6 1  J .  Omura  and  T.  Kailath,  "Some  Useful  Probability  Distribution,1 
Technical  Report  No.  7050-60,  Stanford  Univ. ,  Sept.  1965. 

[17] r.  S.  Huang,  "Some  Notes  on  Film-Grain  Noise,"  Appendix  14, 
"Restoration  of  Atmospherically-Degraded  Images,  "  NSF 
Summer  Study  Rept. ,  Woods  Hole,  Mass.,  pp.  105-109,  1966. 

[18] D.  H.  Kelly,  "Systems  Analysis  of  the  Photographic  Process 
I.  A  Three-Stage  Model,  "  J.  Opt.  Soc.  Am. .  Vol.  50,  p.  269, 
i960. 

[19]  H.  W.  Lorber,  "A  Theory  of  Granularity  and  Bleaching  for 
Holographic  Information  Recording,  "  IBM  Res.  Dev.  ,  Vol.  14, 
No.  5,  Sept.  1970. 


62 


— - 


Chapter  3 


ES  TIMA  riON 

In  this  chapter  the  restoration  of  images  degraded  by  film-grain 
noise  is  considered  within  the  context  of  estimation  theory.  In 
section  3.  1  we  investigate  the  applicability  of  some  existing  estima¬ 
tors  to  film-grain  noise  restoration.  Next,  in  section  3.2,  a  linear 
minimum  mean  square  error  (LMMSE)  filter  based  on  non-stationary 
first  order  statistics  of  an  image  is  developed.  Finally  in  section  3.  3, 
this  filter  is  applied  to  one-dimensional  and  two-dimensional  signals 
and  its  performance  is  compared  with  conventional  discrete  Wiener 
fi  Iter. 

3.  1  Survey  of  Some  Estimators 

To  compare  and  study  the  relevance  of  some  existing  estima¬ 
tors  to  the  problem  of  film-grain  noise  restoration,  let  us  establish 
some  notation.  Let  us  denote  the  signal  to  be  estimated  by  the 

random  sequence  x. ,  x_,  , . . ,  x,  ,  . . . ,  x_  where  x,  is  the  signal  at  time 
1  2  k  N  k 

or  pos i  tion  k.  Similarly  let  y . ,  y-,  .  . .  ,  y  ,  .  .  .  ,  y  denote  the  observ- 

i  c  k  In 

ation  sequence  with  y^  the  observation  at  time  or  position  k.  Usually 
y^  is  a  function  of  x^  and  a  noise  term  n^  which  has  caused  the 
degradation. 
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We  can  generally  write 


yk  =  f(xk»nk»k)  (3.1-1) 

where  f  is  a  deterministic  function.  Equation  (3.  1-1)  is  often 
referred  to  as  the  observation  model  and  as  we  shall  discuss,  the 
implementation  of  the  estimator  designed  to  estimate  x  based  on 

K. 

observation  can  vary  greatly  in  complexity  depending  on  the  nature 
of  the  function  f. 

One  can  classify  various  estimators  as  either  batch  processors 
or  recrusive  estimators.  A  batch  processor  processes  all  of  the 
observations  yj,...,y  at  the  same  time  and  outputs  the  estimates  of 
Xj,...,x^  all  at  once.  On  the  other  hand,  a  recursive  estimator 
provides  an  estimate  for  x^  based  only  on  a  small  number  of  obser¬ 
vations  and  past  estimates.  The  procedure  is  then  repeated  to 
produce  an  estimate  for  x^  j  and  so  on. 

If  can  be  shown  [l  ]  that  for  a  large  class  of  cost  functions  the 
best  estimate  (thebest  in  terms  of  minimizing  the  average  cost)  of 
Xk  using  observations  y^ , . . . ,  y  is  given  by 

A 

*k  *  Et\lyi . 

where  E  denotes  the  ensemble  average  operator.  In  the  above  equa¬ 
tion  if  l  =  k  then  xk  is  called  the  filtered  estimate.  If  i<k  the  esti- 

A 

mate  is  called  the  predicted  estimate  and  if  £>k,  xfc  is  called  the 
interpolated  estimate  of  x^.  In  this  last  case  where  the  estimator 


f 
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uses  observations  in  the  future  to  estimate  the  signal  at  the  present, 
the  estimator  is  called  non-causal.  Obviously  if  a  high  degree  of 
correlation  exists  within  the  signal  samples  and/or  within  the  noise 
samples  then  a  non-causal  estimator  always  gives  a  better  estimate 
than  a  causal  estimator  because  it  uses  more  of  the  available 
information.  Clearly  a  batch  estimator  as  we  have  defined  it  in  this 
section  is  a  non-causal  estimator.  In  most  situations  in  image 
processing,  the  quantity  of  data  to  be  processed  is  enormous  and 
there  are  difficulties  in  implementing  a  batch  estimator.  To 
illustrate  this  point,  consider  a  digitized  image  consisting  of  256  by 
256  pixels.  A  general  batch  estimator  will  have  to  process  over 
65,  000  observations  at  the  same  time  to  produce  an  estimated  image. 
With  the  size  of  most  digital  computers  this  causes  storage  and 
computational  difficulties.  By  comparison,  a  recursive  estimator 
may  or  may  not  be  causal,  but  in  either  case  is  usually  easier  to 
implement  because  it  uses  a  relatively  small  amount  of  data  at  any 
given  time  to  produce  an  estimate. 

Both  recursive  and  batch  estimators  can  be  subdivided  into 
linear  and  nonlinear  estimators.  The  linear  estimator  produces  an 
estimate  which  is  a  linear  combination  of  the  observations. 

As  stated  earlier,  the  optimal  Bayesian  estimate  for  a  large 
class  of  cost  functions  is  given  by 
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(3.1-3) 


=  E[x|y] 

where  y  denotes  the  observation  to  be  used  for  the  estimate  and  x  is 
either  the  signal  at  time  k  or  the  entire  signal  sequence  x^,  ...  ,  x^ 
depending  on  whether  the  estimator  is  recursive  or  a  batch  estimator. 
Using  the  definition  of  ensemble  average  we  can  rewrite  equation 
(3.  1-3)  as 

A 

X  =  J  xP(x|y)dx  (3.1-4) 

x 

where  P(x|  y)  is  the  a  posteriori  probability  density  of  x. 

One  can  show  that  the  estimate  given  by  eq.  (3.  1-3)  is  the  best 
estimate  regardless  of  the  functional  relationship  between  the  obser¬ 
vation,  the  signal  and  the  noise.  However,  if  the  function  f  in  the 
observation  model  of  eq.  (3.  1-1)  is  nonlinear  in  signal  and  noise  then 
the  calculation  of  the  probability  density  function  required  to  carry 
out  the  estimate  of  eq.  (3.  1-4)  can  become  very  difficult.  The  general 
problem  of  implementing  the  optimal  estimator  using  nonlinear 
observations  still  remains  unsolved.  The  most  recent  work  done  in 
this  area  is  by  Naraghi  [2l  who  proposes  a  sub-optimal  algorithmic 
estimation  method  applicable  to  nonlinear  observations. 

Even  with  many  simplifying  assumptions,  estimation  of  images 
degraded  by  film-grain  noise,  still  involves  nonlinear  observations. 

As  discussed  in  the  last  chapter,  D^,  the  observed  optical  density  of 
the  degraded  image  is  given  by 
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'  XT  *v 


(3.  1-5) 


* 

L : 


D  =  D  +  kD  . n 
oss 


where  denotes  the  recorded  image  in  the  absence  of  noise.  In  the 
above  equation  we  have  ignored  all  the  optical  and  chemical  degrada¬ 
tions  as  well  as  the  effect  of  the  nonlinear  H  and  D  curve.  We  can 
see  that  even  for  this  very  simplified  case  the  observation  is  quite 

nonlinear  and  the  optimal  estimate  of  D  as  given  by  E[D  |d  1  would 

s  so 

be  difficult  to  implement.  For  this  reason  various  investigators 
have  tried  to  find  alternative  solutions  to  film-grain  noise  restoration. 
In  the  remainder  of  this  section  we  consider  some  of  the  more  recent 
approaches  to  this  problem  to  provide  a  background. 

1 .  Maximum  A  Posteriori  (MAP)  Estimator 

Recently  Hunt  [3]  has  considered  image  restoration  based  on  a 
model  given  in  Fig.  3.  1-1.  This  model  is  a  much  simplified  version 
of  the  model  shown  in  Fig.  2.  8-1  of  Chapter  2,  Here  the  signal 
dependence  of  the  noise  is  ignored  and  the  main  emphasis  is  placed 
on  the  nonlinear  nature  of  the  function  g(*  ).  From  Fig.  3.  1-1  the 
vector  observation  model 


y  =  g([H]*x)  +  n 


(3.1-6) 


canbe  written.  By  ignoring  the  signal  dependence  of  the  noise,  we  see  that 
the  nonlinearity  in  the  observation  model  of  eq.  (3.  1-6)  does  not 
include  any  cross -multiplication  term  involving  the  signal  and  the 
noise,  as  was  the  case  in  eq.  (3.  1-5),  but  rather  is  completelv  in 


terms  of  the  signal  component  only. 


Hunt  argues  that  the  Bayesian  MMSE  estimate  of  x  given  by 

E[x|y^  involves  calculating  the  a  posteriori  density  P(x|y),  which  is 

not  simple  due  to  nonlinear  function  g(«  ),  and  instead  decides  on  the 

MAP  estimate  of  x,  This  estimate  is  the  mode  of  the  a  posteriori 

density.  Unlike  E  [  x  |  yl  ,  which  is  the  mean  of  P(xj  y),  the  MAP 

estimate  does  not  explicitly  require  knowledge  of  P(x|  y). 

In  deriving  his  estimator  equations.  Hunt  assumes  that  the 

noise  vector  n  in  (3.  1  -6)  is  wide-sense  stationary  and  multivariate 

Gaussian  with  stationary  variation  about  a  nonstationary  mean  x  . 

'  m 

Thus,  the  first  two  moments  of  x  can  be  expressed  as: 


Ex  =  x 


m 


E[x-x  ]  fx-x  ]T  =  R — 
m  m  xx 


(3.1-7) 
(3.  1-8) 


where  x  is  a  variable  vector  and  R — is  a  Toeplitz  matrix.  This  is 
m  xx 

a  very  reasonable  description  of  most  images  and  is  superior  to  the 
assumption  of  wide  sense  stationary  [4]  . 

The  MAP  estimate  as  derived  by  Hunt  has  the  form 


x,,,„  =  x  + 
MAP  m 


(3.1-9) 


where 
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(3.  1-10) 


and  where 


b 


[hi: 


"map 


(3.1-11) 


Equations  (3.  l-9)-(3.  1-11)  are  a  set  of  nonlinear  vector  equatione 

requiring  a  great  deal  of  effort  to  solve.  For  a  100  x  100  image, 

4  4 

R — is  10  x  10  matrix.  Hunt  uses  Picard's  iteration  method  to 
xx 

iteratively  solve  eq.  (3.1-9). 

An  obvious  drawback  to  Hunt's  estimate  is  the  over  simplification 
of  his  model  as  given  in  Fig.  3.1-1  on  which  the  estimate  is  based. 

In  situations  when  the  operation  is  on  a  linear  region  of  the  H  and  D 
curve  (see  Fig.  1.2-1  of  Chapter  1)  the  nonlinear  function  g  of 
eq.  (3.  1-6)  can  be  well  approximated  by  a  straight  line.  Hunt's 
emphasis  in  the  observation  model  is  on  the  nonlinearity  of  this 
function  rather  than  on  the  signal  dependence  of  the  noise.  To  obtain 
a  solution  in  a  reasonable  time,  the  MAP  estimator  requires  a  very 
large,  fast  digital  computer. 
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2.  Wiener  Filter 

The  Wiener  filter  in  the  Fourier  domain  is  a  classical  linear 
estimation  technique  applicable  to  stationary  signals.  Walkup  and 
Choens  [5"|  use  the  principle  of  Wiener  filter  along  with  the  obser- 
vation  model 


1/3, 


D  (f,D)  =  D  (F,Tl)+kD  '  (f ,  T))*  n(p,  T)) 

OSS 


(3. 1-121 

To  derive  a  linear  minimum-mean-square  error  filter  given  by 


W(U)lfU)2)  = 


»D  D  '“l >  “2 * 
s  s 


2  2/3 

*D  D  (U,1*U,2)  +  k  1 

s  s 


(3.  1-13) 


where  ^  is  the  spatial  power  spectrum  of  the  signal,  E  is  the 


s  s 

expectation  operator  and  n  is  taken  to  be  white  noise  with  zero  mean 
and  unit  variance. 

The  filter  W  given  by  the  above  equation  assumes  images  to  have 
stationary  first  and  second  order  statistics.  This  assumption  is  not 
very  reasonable,  particularly  for  first  order  statistics.  In  addition, 
no  optical  and  chemical  degradations  are  included  in  the  observation 
model  of  eq.  (3.  1-12)  and  filter  W  will  only  filter  out  the  noise  and 
will  not  compensate  for  other  degradations. 

3.  2  Optimal  Discrete  LMMSE  Filter 

The  object  of  this  section  is  to  develop  an  optimal  discrete 
linear  minimum-mcan-square  error  filter.  Before  doing  so  however 
we  would  like  to  list  some  desirable  features  for  such  a  filter: 
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a)  As  discussed  in  chapter  2  and  again  in  section  3.  1,  the 
observed  optical  density  can  be  modeled  by 

D  =  D  +  kDP*n  (3.2-1) 

os  s 

if  only  the  effects  of  film-grain  noise  are  considered  and  other 

degradation  factors  are  ignored.  Note  that  even  though  the  noise  n 

in  the  above  model  is  statistically  independent  of  the  signal  D  ,  the 

s 

deviation  of  the  observation  D  from  the  signal  D  is  very  much 

o  s 

p 

dependent  on  the  signal  level  through  the  term  kDg  *n.  This  deviation 
is  more  pronounced  when  the  signal  level  is  high  and  is  less  severe 
when  the  signal  level  is  low.  Thus  the  filter  designed  to  estimate  D 

s 

based  on  the  observation  D  should  be  aware  of  this  fact.  In  other 

o 

words  the  filter  should  be  adaptive  in  the  sense  that  it  should  change 
its  characteristics  to  account  for  the  presence  of  higher  noise  when 
it  is  operating  in  a  region  where  the  expected  signal  level  is  high. 

b)  As  pointed  out  in  the  last  chapter,  the  process  of  recording 
an  image  on  film  involves  optical  and  chemical  degradations  in 
addition  to  film-grain  noise.  We  would  therefore  like  the  restora¬ 
tion  filter  to  compensate  for  these  degradations  while  filtering  out 
the  noise. 

c)  Most  existing  restoration  filters  are  designed  based  on  the 
assumption  that  images  are  characterized  by  stationary  first  and 
second  moments.  As  discussed  before,  this  is  not  a  very  realistic 


?2 


assumption,  particularly  in  the  first  moment.  To  demonstrate  this. 
Cannon  and  Hunt  give  the  following  argument  [4]  :  Suppose  one  uses 
several  thousand  pictures  from  Californis  drivers'  licenses  in  order 
to  calculate  an  ensemble  mean  image.  Clearly  such  an  ensemble 
mean  image  would  not  consist  of  a  uniform  shade  of  gray  indicating 
stationarity.  More  likely,  the  mean  image  would  consist  of  an 
elliptical  region  where  the  face  is  expected  to  be  and  probably  some 
dark  spots  where  the  eyes,  nose  and  other  facial  features  are 
expected  to  be.  Thus  images  are  generally  non-stationary  in  first 
order  statistics,  and  we  would  like  to  utilize  this  in  the  derivation  of 
the  filter. 

d)  Since  we  are  to  process  digitized  images  on  digital  computers 
we  would  like  the  filter  to  be  discrete. 

The  first  two  of  the  above  desired  properties  can  be  immediately 
incorporated  in  the  restoration  filter.  In  the  model  of  Fig.  2.8-1  for 
the  imaging  system,  the  first  and  the  last  segments  of  the  model 
accounts  for  various  degradations  other  than  noise.  In  the  middle 
segment  the  nonlinear  noise  is  inserted  to  reflect  the  signal- 
dependence  term  in  eq.  (3.2-1).  Thus  if  we  derive  the  restoration 
filter  in  accordance  with  this  model,  then  both  the  a  and  the  b 
features  mentioned  in  this  section  will  be  incorporated  in  the  filter 
automatically. 

To  obtain  a  discrete  filter,  we  must  sample  the  continuous  model 


of  Fig.  2.8-1.  Figure  3.2-1  shows  the  block  diagram  of  the  equival¬ 
ent  discrete  model.  In  this  figure  I  is  a  column  vector  of  size  N 
representing  the  incident  intensity  which  is  the  input  to  the  imaging 


system.  The  M  XN  blur  matrix  represents  the  combined  effects 
of  linear  space-invariant  blurs  b^x.y)  and  b^x,  y)  of  the  continuous 
model.  It  is  assumed  that  the  operation  is  on  the  linear  region  of  H 
and  D  curve  and  thus  the  nonlinear  function  g(« )  of  Fig.  2.8-1  is 
approximated  by  a  straight  line.  In  the  discrete  model  the  slope  of 
this  line  is  taken  to  be  a  and  constant  vector  B  contains  the  intercept 
point  of  this  line  for  its  elements.  Vector  N  represents  the  noise. 

It  is  multiplied  by  a  M  X  M  matrix  F  which  is  diagonal  and  has  the 
form 


CFl=  k 


(de,P 


0 


(3.2-2) 


0 


(dM)P  _J 
e 


where  k  is  the  scanning  constant  and  d^  is  the  j  element  of  vector 

D  .  Noise  N  is  taken  to  be  white,  multivariate  Gaussian  whose  first 
e 

two  moments  are  given  by 


E(N)  =  0  (3.2-3) 

CNN  =  E(N.NT)  =  61  (3.2-4) 

where  [ll  denotes  the  identity  matrix  and  E  is  the  ensemble  average 
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operator.  Note  that  matrix  [f!  is  structured  to  conform  /ith  the 


scalar  eq.  (3.2-1).  The  R  X  M  matrix  H ^  represents  the  combined 

effects  of  linear  blurs  b^(x,  y)  and  b^(x,  y)  in  the  continuous  model. 

Finally  vector  D  is  the  observed  degraded  signal.  The  restoration 
o 

filter  should  provide  an  estimate  for  the  input  signal  I  based  on 
observation  and  some  a  priori  statistical  information.  Denoting 
this  filter  by  an  N  X  R  matrix  [w"],  the  estimate  will  be  given  by 

A 

T  =  tW]D  .  (3.2-5) 

It  has  been  shown  [6"|  that  the  optimal  filter  has  the  form 

=  tc75  itc55  r1  (3.2-6) 

o  o  o 

where  the  criterion  of  optimality  is  to  minimize  the  expected  value  of 
the  square  of  the  estimation  error  given  by 

A  * 

e  =  Ea-T)Ta-T)  .  0.2-7) 

Here  (C— —  )  and  (C—  —  )  are  the  crosB -covariance  of  the  signal  and 
o  00 

the  observation  and  autocovariance  of  the  observation  respectively. 

It  should  be  noted  that  the  filter  fw ]  of  eq.  (3.2-6)  is  a  linear  non- 
causal  batch  processor.  No  assumption  of  stationarity  is  imposed  in 
the  derivation  of  this  filter,  and  the  general  form  as  given  by  eq. 
(3,2-6)  is  independent  of  any  underlying  observation  model.  However, 
the  evaluation  of  the  two  covariance  matrices  requires  the  knowledge 
of  an  observation  model.  We  now  devote  the  rest  of  this  section  to 
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the  evaluation  of  this  filter  according  to  the  discrete  model  of  Fig. 
3.2-1. 

A.  Evaluation  of  [C—  —  ] 

o 

Using  the  definition  of  cross  covariance  we  have 

T  r 

[C— —  ]  =  E(f  D  )  -  r  D 
I  D  J  o  mo 

o  m 


(3.2-8) 


where  I  and  D  are  the  nonstationary  mean  of  the  signal  and  the 
m  o 

m 

observation  respectively.  From  Fig.  3.2-1  we  can  write 


D  =  [h-Wh.Ii  +  B  +  [FlN) 

O  LX. 


(3.2-9) 


Therefore 


E(I  Dq)  =  EtKatH^I  +  B  +  [F]N  )•  tH2f]  .  (3.2-10) 


Now  let  us  assume  that  the  signal  I  and  noise  N  are  independent, 


e. 


fr  f}(ll ’  l2*  •  •  • '  lN:nl '  n2 . nN)  =  fI(l  1 . lN)#  fN(nl '  *  *  * '  nN) 

(3.2-11) 

where  f  is  the  joint  probability  density  function  of  I  and  N.  Using 

r  tt 

eq.  (3.2-11)  in  eq.  (3.2-10)  we  see  that 

p 

E(f  D  )  =  aE(ir  )[H.lT(H,lT+r  BT[H,1T.  (3.2-12) 
o  i  £  m 

We  now  calculate  the  second  term  in  eq.  (3.2-8)  namely 
_T 

r  *D  .  From  eq.  (3, 2-9)  we  have 
m  o 

m 
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where 


T 

77  77  r--.T, 


^N'N'1  "  E^F1NN  1 


(3.2-20) 


in  the  above  equation.  Equation  (3.2-2)  and  (3.2-4)  indicate  that 


[C . 1  has  the  following  form 

N'N' 


E(d‘)2P 

e 


2  2P 
E(d  )  ^ 
e 


(3.2-21) 


L  ‘  e'^>2pJ 

i  th  — 

where  d  is  the  j  element  of  vector  D  of  Fig.  3.2-1.  The  second 
6  _  _ T  6 

term  in  equation  (3.2-17)  is  D  ‘D  and  this  can  be  calculated 

o  o 

m  m 

using  eq.  (3.2-9)  as  shown  below 


_  ,  2  -  -1  v _ 1 

Do  -Ro  "M0  +BB  * 

m  m  _ T _ T 

a  [H.  1 1  B  +  aB  I  kn.  [H,  r.  (3.2-22) 

l  m  mi/  c 

Finally  using  eq.  (3.  2-19)  and  (3.  2-22)  in  eq.  (3.  2-17)  we  obtain 

[crrn  1  =  ^rr1  tHi'|T  +  S'nO (3-2-231 

o  o 

These  expressions  for  [C—  —  land  [C^  ^  1  indicate  that  the 

o  o  o 

filter  [W 1  has  the  form 


[Wl  =  a[Crrl[H1lr(a2Ih1ltCrrltM11r  +  tCN,N,l)'  [H2r‘.  (3.2-24) 
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The  filter  [w ]  appears  to  have  the  form  of  a  conventional 
discrete  Wiener  filter  except  for  the  presence  of  the  blur  matrix 
[H2],  and  more  importantly,  the  structure  of  matrix  .  From 

Fig.  3.2-1  and  eq.  (3.2-21)  we  see  that  ia  a  diagonal  matrix 

whose  diagonal  is  given  by 

Diagonal  of  fc  ,]  =  ak2 •  E^H^f  +  B)2P.  (3.2-  25) 

In  Chapter  2  we  discussed  experimental  results  indicating  that  the 
exponent  P  in  the  above  equation  is  less  than  j  and  is  more  likely  in 
the  range  0.  3-0.4.  If  for  the  sake  of  simplicity  we  take  P  to  be  |  we 
see  that  eq.  (3.2-25)  reduces  nicely  to 

Diagonal  of  [C ^ ( ^ ,  1  =  ak  [H j  "]  I  ^  +  ak2B,  (3.2-26) 

Equation  (3,2-26)  points  out  that  the  diagonal  of  the  effective  noise 
covariance  matrix  is  given  by  the  scaled,  blurred  non¬ 

stationary  a  priori  mean  of  the  signal  I .  It  is  exactly  this  feature 
that  enables  the  filter  to  operate  adaptively  based  on  a  priori  informa¬ 
tion.  Using  the  nonstationary  expected  value  of  the  signal  I  to  anti¬ 
ng 

cipate  the  level  of  the  signal,  the  filter  recognizes  regions  where  the 

signal  level  is  expected  to  be  high  as  regions  of  high  noise  and  filters 

accordingly.  Conversely  if  the  signal  level  is  not  expected  to  be 

high  in  a  region,  then  the  filter  does  not  filter  as  much.  This  is  the 

adaptive  quality  that  we  wanted  to  incorporate  in  the  restoration  filter. 
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In  comparison,  the  diagonal  elements  of  the  covariance  of  the  noise 


* 


matrix  in  conventional  discrete  Wiener  filter  are  all  equal.  Thus 
the  filter  assumes  equal  noise  in  all  the  regions,  which  as  we  have 
seen  is  not  the  case  for  images  degraded  by  film-grain  noise. 

Before  applying  this  filter  to  some  one -dimensional  and  two- 
dimensional  signals,  we  conclude  this  section  by  deriving  an  expres¬ 
sion  for  the  mean  square  error  (mse)  as  given  by 

A  * 

mse  =  Trace{E[(F-Fj(F-l") r”l  }  .  (3.2-27) 

Using  eq.  (3.2-5)  we  have 

mse  =  Trace{E[(I-  [w]Do)(I-rw]Do)Tll  (3.2-28) 

and  carrying  out  the  above  expectation  we  get 

mse  =  Trace{[Cj-j-  1-  2 twite,- D  1r  +  tcrT)  -|[w-|l 

°  °  (3.2-29) 

where  all  the  terms  in  the  above  equation  have  previously  been 
defined. 

3.  3  Experimental  Results 

As  a  preliminary  evaluation  of  the  performance  of  the  filter 
derived  in  the  last  section  we  first  applied  it  to  a  one-dimensional 
signal.  Figure  3.  2 -2a  shows  the  ideal  signal  corresponding  to  the 
input  signal  I  of  Fig.  3.2-1.  This  signal  is  obtained  from  one  line  of 
a  typical  digitized  image,  and  the  size  of  vector  I  is  taken  to  be  128. 
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We  then  degraded  this  signal  in  accordance  with  the  model  of  Fig. 

3.2-1.  First,  the  vector  I  was  blurred  with  matrix  [Hjl  which  was 

a  Gaussian  blur  matrix  with  window  of  5  elements  and  a  n  of  0.9.  The 

result  is  shown  in  Fig.  3.2-2b  and  corresponds  to  the  vector  E  of 

Fig.  3.2-1.  Because  the  constant  a  and  constant  vector  B  act  as  a 

scaling  factors  only,  without  any  loss  of  generality,  they  were  set 

to  one  and  zero  respectively.  Therefore  in  this  simulation,  the 

vectors  E  and  of  Fig.  3.2-1  are  identical.  A  Gaussian  signal- 

dependent  noise  was  then  added  to  vector  D  to  obtain  the  signal 

shown  in  Fig.  3.2-3a.  The  scanning  constant  k  of  eq.  (3.2-2)  was 

taken  to  be  0.3,  corresponding  to  a  fairly  noisy  film  scanned  with  a 

very  small  aperture.  Finally,  vector  D  was  blurred  with  matrix 

r 

[H2]  which  was  a  Gaussian  blur  with  a  window  of  3  elements  and  a  a  of 
.  5  to  yield  the  observed  signal  Dq  shown  in  Fig.  3. 2 -2b.  This 
signal  was  processed  by  the  LMMSE  filter  derived  in  the  last  section 

A 

to  yield  an  estimate  I  for  the  input  signal  I.  The  estimated  signal  is 
shown  in  Fig.  3.2-4b.  Comparing  figures  (a)  and  (b)  of  Fig.  3,2-4 
we  note  that  the  filter  has  filtered  more  in  the  high  signal  levels 
than  the  low  signal  levels.  Overall  the  calculated  mean -square - 
error  was  improved  by  a  factor  2,5. 

Note  that  in  implementing  filter  of  eq.  (3.2-24)  one  needs 
to  know  Im  and  [C— 1-”|,  the  first  two  moments  of  the  signal.  In  this 
work  ]  was  approximated  by  the  covariance  matrix  of  a 
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Markov  process  vector  with  first  order  correlation  of  0.  85.  The 

a  priori  ensemble  mean  of  the  signal  was  estimated  at  each  point  by 

blurring  the  observed  signal  D  with  a  very  large  window  to  obtain  a 

rather  smooth  spatial  average  curve  that  was  then  used  for  I  .  This 

m 

procedure  effectively  approximates  1^  by  a  local  spatial  average  of 

D  . 
o 

For  the  restoration  of  two-dimensional  signals  or  images  one  can 
arrange  the  pixels  of  the  digitized  image  lexographically  in  a  long 
vector  and  then  proceed  in  the  same  manner  discussed  above  for 
processing  one -dimensional  signals  [8]  .  The  problem  with  this 
procedure  is  that  for  a  reasonable  image  of  size  256  x  256  the  result¬ 
ing  vector  would  have  length  65,  536.  Obviously,  the  manipulation  of 
vectors  and  matrices  of  this  size  becomes  horrendous,  even  with 
large  digital  computers.  An  alternative  method  is  to  break  up  the 
image  into  small  blocks  and  to  process  each  sub-block  individually. 
Similarly  one  can  consider  rows  and  columns  of  the  image  as 
separable  one -dimensional  signals  and  sequentially  process  the 
rows  to  obtain  one  estimate  and  then  sequentially  process  the  columns 
to  obtain  a  second  estimate.  The  final  restored  image  is  then  a 
combination  of  these  two  estimates  [6l  -[9l,  with  the  simplest  com¬ 
bination  being  the  average.  Note  that  both  the  block  filtering  or 
sequential  row  and  column  filtering  result  in  suboptimal  estimates. 

In  this  section  we  present  an  example  of  two-dimensional 


restoration  using  the  sequential  row  and  column  processing  approach. 
However  before  doing  so  an  important  point  needs  to  be  mentioned. 
Since  we  assume  the  image  to  have  a  nonstationary  mean  then  the 
a  priori  mean  for  various  lines  of  the  image  are  not  all  equal. 
Equations  (3.2-24)  and  (3.2-26)  point  out  that  every  time  the  mean  of 
a  particular  row  differs  from  the  mean  of  the  previous  row,  a  new 
covariance  of  the  noise  must  he  calculated,  in  turn  requiring 

a  new  filter  [w]  to  be  calculated.  Since  any  change  in 
necessitates  performing  a  matrix  inversion  in  order  to  calculate  the 
filter  [w]  of  eq.  (3.2-24),  this  causes  an  imposing  problem.  This 
problem  is  avoided  by  realizing  that  in  most  typical  images  of 
interest  the  means  of  adjacent  lines  do  not  differ  substantially  unless 
a  sharp  boundary  is  passed  in  going  from  one  line  to  the  other.  Thus, 
the  following  algorithm  is  used  in  processing  two-dimensional  signals: 
When  processing  line  j  we  compare  the  a  priori  mean  for  line  j  with 
the  mean  of  the  previous  line;  if  no  substantial  difference  exists  we 
process  line  j  with  the  same  filter  that  was  calculated  for  line  j-1. 

If,  however,  the  difference  between  the  two  means  is  judged  to  be 
considerable,  then  the  new  mean  is  used  to  calculate  a  new  filter 
which  is  then  used  to  process  line  j. 

Figure  3.2-5  shows  an  example  of  the  application  of  this  filter 
to  images.  Figure  (a)  shows  the  ideal  image.  The  three  density 
levels  are  0,4  for  the  background,  1.  1  for  the  left  hand  square  and 
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1.8  for  the  right  hand  square.  The  density  for  this  image  has  a 
fairly  large  dynamic  range;  thus  the  noise  in  the  degraded  image  is 
quite  signal-dependent.  Figure  (b)  shows  the  degraded  image 
simulated  according  to  the  model  of  Fig.  3.2-1.  In  this  simulation 
the  initial  blur  is  Gaussian  with  a  window  of  5  X  5  elements  and  a  of 
0.  9.  The  final  blur  is  Gaussian  with  a  window  of  3  x  3  elements  and 
a  of  0.  5.  The  change  in  the  a_  priori  mean  of  the  rows  and  columns 
requires  the  calculation  of  four  different  filters,  each  using  a  different 
mean  for  the  matrix  . 

In  Fig.  3.2-5  note  that  the  filter  has  done  more  filtering  in  the 
two  square  areas  where  the  noise  is  considerable.  In  the  restored 
image  there  is  some  degree  of  smoothing.  This  smoothing  is  an 
inherent  property  of  an  estimator  using  mean-square  error  as  the 
optimality  criterion. 

The  performance  of  this  filter  can  be  compared  with  that  of  a 
conventional  Wiener  filter  which  ignores  the  signal-dependence  of  the 
noise.  The  result  of  the  comparison  is  shown  in  Fig.  3.2-6.  Com¬ 
paring  the  square  regions  in  the  two  restored  images  we  see  that  the 
conventional  Wie.ier  filter  does  not  filter  these  regions  as  well  as  the 
other  filter  because  it  assumes  the  noise  to  be  the  same  in  all  the 
regions. 
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Chapter  4 
DETECTION  I 

In  digitizing  photographic  films,  it  is  sometimes  necessary  to 
use  very  small  apertures  in  the  scanning  microdensitometer  in  order 
to  detect  fine  details  in  the  image.  With  these  small  apertures  the 
film-grain  noise  is  so  severe  that  the  detection  of  fine  details  and 
distinction  between  adjacent  areas  of  small  contrast  becomes  virtually 
impossible.  The  noise  can  be  reduced  by  using  a  larger  aperture  for 
the  scanner;  however  doing  this  will  produce  averaging  over  fine 
details  of  interest  in  the  image.  Thus  in  most  cases,  the  photograph 
must  be  digitized  using  a  small  aperture  and  then  improved  by 
effective  enhancement  and  restoration  techniques. 

2 

When  scanned  with  apertures  as  small  as  4  to  25  (urn.)  ,  the 

digitized  image  is  so  degraded  that  conventional  restoration  techniques 

employing  linear  or  nonlinear  estimators  will  have  little  effect.  In 

these  cases  heuristic  but  clever  detection  schemes  taking  the 

properties  of  the  human  observer  into  account  appear  to  be  more 

helpful  than  any  estimation  method.  In  this  chapter  we  analyze  an 

enhancement  method  called  "noise  cheating"  introduced  by  H.  J. 

Zwcig  et  al.  [l]  and  show  that  their  work  can  be  justified  through 

classical  detection  theory.  The  "noise  cheating"  algorithm  is  an 

effective  enhancement  method  for  low  contrast  images  having  a  small 
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dynamic  range. 

To  expand  on  this  algorithm  and  make  it  applicable  to  a  broader 
class  of  images,  and  to  develop  new  and  more  flexible  algorithms  of 
this  type,  we  must  provide  a  justification  for  the  heuristic  nature  of 
the  "noise  cheating"  algorithm.  Without  such  justification,  further 
expansion  on  the  idea  beyond  the  heuristic  procedures  becomes 
impractical.  Thus,  this  chapter  is  devoted  to  analysis  of  "noise 
cheating"  algorithm  within  the  framework  of  detection  theory.  In 
the  next  chapter  we  improve  this  algorithm  so  that  it  is  applicable  to 
a  broader  class  of  images.  We  will  also  develop  a  Bayesian 
detection  method  which  is  considerably  more  flexible  than  the  "noise 
cheating"  algorithm. 

The  "noise  cheating"  method  in  essence  functions  as  a  two- 
stage  detecting  procedure.  A  detailed  analysis  of  this  two-stage 
procedure  is  discussed  in  the  rest  of  this  chapter. 

4. 1  Enhancement  by  Two-Stage  Maximum  Likelihood  Detection 

Assume  that  in  the  absence  of  grain-noise,  the  section  of 
photograph  to  be  scanned  is  composed  of  M  extended  regions  R^,.... 

all  of  them  of  much  larger  size  than  the  scanning  aperture  and 
each  of  them  having  a  uniform  density.  Denote  by  D, .....  D  the 
density  levels  of  these  M  regions  and  assume  the  image  is  extremely 

low  contrast  so  that  values  of  D......D  are  very  close. 

1  M 


Suppose  this  film  is  scanned  with  a  small  square  aperture  of 
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size  2a  x  2a.  If  a  is  on  the  order  of  1  -  3  [im,,  then  the  M  regions 
will  no  longer  be  distinguishable  from  one  another  in  the  digital 
image  due  to  severity  of  the  noise  and  the  low  contrast.  Thus  some 
sort  of  enhancement  is  necessary. 

To  begin  our  analysis  of  the  "noise  cheating"  algorithm,  we 
start  with  a  set  of  simplifying  assumptions.  Though  unrealistic, 
these  assumptions  will  facilitate  the  understanding  of  the  nature  of 
the  algorithm  and  once  this  understanding  is  reached  these  assump¬ 
tions  will  be  dropped. 

The  first  assumption  is  that  we  know  a  priori  that  the  image 

consists  of  exactly  M  distinct  regions,  even  though  we  cannot  make 

out  all  of  these  M  regions  in  the  degraded  image.  We  also  know  the 

density  levels  of  these  regions;  thus  we  assume  a  priori  knowledge  of 

values  for  D . D  . 

t  M 

Based  on  each  single  reading  of  the  scanner  (each  single  reading 
of  the  scanner  corresponds  to  a  pixel  in  the  digitized  image)  we 
want  to  decide  if  the  reading  was  taken  when  the  densitometer 
aperture  was  completely  in  one  of  the  M  regions  or  if  it  was  taken 
with  the  aperture  overlapping  two  or  more  of  the  regions.  If  this 
decision  can  be  made,  then  by  changing  the  value  of  each  observed 
pixel  to  the  density  of  the  detected  region,  some  of  the  noise  effects 
should  be  suppressed. 

To  achieve  this,  we  formulate  the  problem  as  the  following  M+l 
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hypothesis  detection  problem.  Corresponding  to  each  single  reading 

produced  by  the  scanner,  we  denote  by  the  hypothesis 

that  the  reading  was  taken  with  the  scanner's  aperture  completely  in 

regions  R,,R_,  ...,R  respectively.  Let  H  ,  denote  the  hypothesis 
It.  M  M+t 

that  the  reading  was  taken  with  the  aperture  overlapped  by  two  or 
more  regions.  Figure  4.1-1  illustrates  these  hypotheses. 

As  we  discussed  in  Chapter  2,  the  output  of  the  scanner  is  a 
function  of  the  mean  density  of  the  region  scanned  and  the  granularity 
noise.  Denoting  the  scanner's  output  by  y,  we  write 

y  =  g(D,n)  (4.1-1) 


corresponding  to  hypothesis  HM  + 


Figure  4.  1-1.  Position  of  the  scanning  aperture  under 

various  hypothesis.  .  j 

i 
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i  =  1 


M 


(4. 1-3) 


£(y  lHi)  =  -79= 
where 


Jtna. 


>-&y-Di)2/2af] 


a.  =  kD. 

i  i 


1/3 


(4.1-4) 

However  f(y|H^+^)  is  far  more  complicated.  In  general  we 
can  write 

-([r-(a1D,«2D2+...«MDM)]2 /z4+1) 


f(y^HM+l)  =  7^ 


M+l 


(4.  1-5) 


where  0  sa.  <1.  The  exact  form  of  this  conditional  density  depends  on 
a. 's  which  in  turn  depends  on  the  number  of  regions  and  the  amount 
of  overlap  in  each  as  shown  in  Fig.  4. 1-1.  It  is  clear  that  no  unique 
expression  for  f (y  |  ^ )  can  be  written  because  there  are  an  infinite 

number  of  expressions  for  this  conditional  density  depending  on 
different  combinations  of  QL^s.  In  fact,  the  set  containing  all  the 
possible  expressions  for  this  density  is  uncountable.  Therefore, 
carrying  out  the  maximum  likelihood  detection  test  as  indicated  by 
eq.  (4.  1-2)  becomes  impossible. 

To  avoid  this  problem,  let  us  consider  a  suboptimal  detection 

» 

procedure.  We  perform  the  detection  in  two  stages.  In  the  first 
stage  we  ignore  the  overlap  hypothesis  H^  +  j  and  assume  that  all  the 
readings  were  taken  with  the  aperture  completely  in  one  of  the  M 
regions.  We  then  perform  M  hypothesis  maximum  likelihood 
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detection  on  the  image  based  only  on  hypotheses  H, , . . . , H  . 

■I  M 

In  the  second  stage  we  examine  the  detected  image  and  in  areas 
where  hypothesis  H^  +  j  appears  very  likely  to  have  occurred  (i.e. 
the  boundary  of  the  M  regions)  we  refine  the  details  of  the  image 
according  to  another  algorithm. 

The  net  effect  is  that  we  accurately  define  the  coarse  features  of 
the  image,  i.e.  the  rough  boundaries  of  the  M  extended  regions,  in 
the  first  stage  and  sharpen  the  edges  in  the  second  stage. 

4.  2  Detection  by  Quantization 

In  this  section  we  consider  the  first  stage  of  the  algorithm  just 
discussed  and  see  how  it  can  be  implemented  by  a  simple  quantization. 

In  the  previous  section  we  made  the  assumption  that  there  is  a 
small  difference  between  the  density  levels  of  different  regions.  We 
can  therefore  rewrite  cq.  (4.  1  -3)  as 


«d'V  =  7jjrr  e'c,y-Di,2/2"  1  i-i . m 


(4.2-1) 


where  a.  has  been  replaced  by  n  .  We  can  choose  0  to  be 


a  =  k(D) 


(4.2-2) 


where  D  may  arbitrarily  be  taken  as  the  mean  value  of  the  D  's.  Thus 

l 

we  have  used  the  low  contrast  of  the  image  to  omit  the  dependence  of 
:7  on  the  D.'s  and  thus  in  effect  neglect  the  signal-dependence  of  the 


noise. 
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Without  loss  of  generality,  we  assume  that  D,  <D  <...<D  , 

12  M 

It  can  now  easily  be  shown  that  the  maximum  likelihood  detection 
rule  of  selecting  if 

f(y I  Hfc)  >  f(y|H.)  i  =  1 . M  (4.2-3) 

i  *  k 

is  the  same  as  selecting  if  and  only  if 


Lk<y<Lk+l 


where  and  L^+1  are  given  by 


Lk  = 


Jkfl 


Dk  *  Dk-1 


2 

D.  +  D 


k+1 


(4.2-4) 


(4.2-5) 

(4.2-6) 


These  equivalent  statements  become  clear  by  a  simple  inspection 
of  Fig.  4.2-1.  Equations  (4.2-3)  to  (4.2-6)  also  point  out  an  interest¬ 
ing  fact.  If  we  quantize  each  pixel  of  the  degraded  image  with  a 
quantizer  whose  quantization  levels  are  placed  at  density  levels  D., 
i  =  1 ,  . . . ,  M  and  whose  decision  levels  are  placed  at  half  way  between 
the  density  levels  at 

D.  +  D.  j 

L.  =  1  2  l~  i  =  2,  . . . ,  M  ,  (4.2-7) 

then  in  effect  we  perform  the  maximum  likelihood  detection  as 
described  by  eq.  (4.2-3).  Therefore  the  first  stage  of  detection  can 
be  implemented  by  quantization  of  the  degraded  image  with  the 


quantizer  described  above 


oo 


Let  us  now  evaluate  the  performance  of  the  quantizer  by 
examining  the  quantized  image  at  the  end  of  the  first  stage.  Suppose 
in  digitizing  the  image  we  select  the  2a  X  2a  aperture  of  the  scanner 
such  that  the  resulting  noise  standard  deviation  as  given  by 

°  '  TSTTJT  10,1/3  ,4-2-81 

is  no  more  than  1/4  of  the  difference  between  the  two  closest  D. 's. 

t 

Therefore  all  D^'s  would  be  at  least  4o  apart.  Selection  of  the  aper¬ 
ture  under  the  above  condition  would  thus  imply  that  the  decision 
levels  of  the  quantizer  used  in  the  first  stage  would  be  at  least  2o  to 
each  side  of  their  corresponding  quantization  levels  as  seen  in  Fig. 
4.2-2. 

Since  pixels  in  the  degraded  image  belonging  to  region  R.  are 

Gaussian  distributed  with  mean  D.  and  standard  deviation  o  ,  then 

roughly  95%  of  the  readings  taken  by  the  densitometer  from  region 

will  be  in  a  band  2a  to  each  side  of  D.  and  therefore  will  be  sub- 

i 

sequently  quantized  to  D^. 

Due  to  the  properties  of  Gaussian  distribution  [3],  this  means 
that  after  the  first  stage  of  detection.  95%  of  the  interior  points  of 
each  region  will  be  correctly  detected  and  restored. 

4.  3  Probability  of  Correct  Detection  in  the  First  Stage 

In  the  previous  section  we  saw  that  with  a  probability  of  0.  95  the 
interior  points  of  uniform  density  regions  are  correctly  detected  and 
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Figure  4.  2-2.  The  quantizer  used  in  the  first  stage  of  the 
detection. 
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restored  to  the  right  density.  These  results  are  conditioned  on  the 
selection  of  an  aperture  for  "he  scanner  so  that  the  noise  standard 
deviation  a  is  no  more  than  1  /4  of  the  difference  between  two  closest 
D.'s. 

i 

From  eq.  (4.2-8)  we  see  that  the  standard  deviation  of  the  noise 
is  inversely  proportional  to  the  scanning  aperture.  Suppo:  e  we 
digitize  the  image  with  a  larger  aperture  such  that  the  resulting  noise 
standard  deviation  is  only  1/6  of  the  difference  between  the  two 
closest  D/s,  This  means  that  the  decision  bands  of  the  quantizer 
now  extend  to  at  least  3a  to  each  side  of  the  quantization  levels.  Due 
to  the  Gaussian  distribution  of  the  scanner  output,  the  probability  of 
correct  detection  for  the  interior  points  of  each  region  increases  to 
0.  97  with  this  larger  aperture. 

From  the  above  argument  it  is  clear  that  as  the  aperture  size 
is  increased,  the  distance  between  the  quantization  levels  increases 
(in  terms  of  a)  and  the  probability  of  correct  detection  increases  for 
the  interior  points.  There  is  a  tradeoff,  however,  because  the  large 
aperture  will  average  over  a  large  area  to  produce  a  single  reading, 
thus  destroying  some  of  the  fine  detail  at  the  boundaries  of  the  M 
regions. 

A  smaller  aperture  preserves  more  information  about  the  edges, 
but  also  decreases  the  probability  of  correct  detection  for  the 
interior  points.  To  illustrate  this  point,  suppose  we  use  a  smaller 


aperture  resulting  in  a  noise  o  which  is  1/2  the  difference  between 


the  two  closest  D^'s.  Again  following  the  same  line  of  argument,  we 
see  that  the  decision  bands  of  the  quantizer  used  in  the  first  stage  are 
now  only  2o  wide  or  one  a  to  each  side  of  the  quantization  levels. 
Therefore  the  probability  of  correct  detection  for  the  interior  points 
is  only  0.  67. 


A  compromise  can  be  reached  by  using  two  different  size  aper¬ 
tures.  A  large  aperture  of  size  2a  X  2a  can  be  used  in  the  first  stage 
of  the  detection.  The  size  of  this  aperture  will  be  selected  so  that  the 
distance  between  any  two  D.'s  is  at  least  4 a  and  thus  the  probability  of 
correct  detection  for  the  interior  points  is  a  high  0.  95. 

As  we  said  before,  the  second  stage  of  the  detection  is  designed 

to  compensate  for  the  omission  of  hypothesis  H.  ,  in  the  first  stage. 

M+t 

Recall  that  this  situation  occurs  when  the  aperture  overlaps  two  or 
more  of  the  extended  regions.  Clearly  this  hypothesis  is  the  most 
probable  when  the  scanner  is  digitizing  areas  around  the  boundary 
of  the  M  extended  regions. 

Since  we  have  detected  the  interior  points  of  each  region  at  the 
end  of  the  first  stage  with  low  probability  of  error,  we  have  roughly 
outlined  the  boundaries  of  the  extended  regions  where  hypothesis 
Hm+i  is  most  likely  to  have  occurred.  The  second  stage  of  the 
detection  should  thus  involve  a  closer  examination  of  the  regions 
which  have  been  designated  to  be  boundary  areas  by  the  first  stage. 


In  the  next  section  we  show  how  these  areas  may  be  redigitized  with  a 
smaller  aperture  to  improve  resolution. 


4. 4  Second  Stage  of  the  Detection 

After  the  first  stage  of  the  detection  and  prior  to  the  second 
stage  involving  boundary  operations,  all  isolated  points  are  removed 
from  the  detected  image.  An  isolated  point  is  defined  to  be  a  pixel 
having  a  value  which  is  different  from  all  of  its  closest  eight 
neighboring  pixels.  In  removing  the  isolated  pixel  the  value  of  the 
isolated  pixel  is  changed  to  conform  with  that  of  its  neighbors.  This 
is  iustified  because  we  assume  the  picture  to  consist  oily  of  extended 
regions  several  times  the  size  of  the  scanning  aperture.  Thus  we  can 
conclude  that  any  isolated  pixel  was  erroneously  detected. 

After  the  first  stage  we  define  a  pixel  to  belong  to  a  boundary 
if  one  or  more  of  its  eight  closest  neighboring  pixels  have  been 
detected  to  have  a  value  different  from  it.  Since  hypothesis  +  1  is 
most  likely  to  have  been  true  at  the  boundaries,  all  the  boundary 
points  need  to  be  reexamined. 

The  reexamination  is  achieved  by  redigitizing  the  photograph 
with  a  smaller  aperture  at  the  areas  which  have  been  designated 
to  be  boundaries  by  the  first  stage.  This  new  data  is  used  to  improve 
the  resolution. 

Suppose  the  smaller  aperture  is  of  size  a  x  a  and  thus  is  only 

1/4  of  the  size  of  the  aperture  originally  used.  This  would  mean  that 
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for  each  single  pixel  produced  by  the  scanner  during  the  first  digitiza¬ 
tion,  we  now  obtain  four  readings.  We  can  now  use  the  newly 
obtained  data  to  sharpen  the  boundary.  An  algorithm  used  by  Zweig 
et  al.  [l]  is  explained  below  by  an  example. 

Referring  to  Fig.  4. 4- la  and  allowing  each  small  square  to  be  a 
pixel  in  the  digitized  image,  suppose  that  after  the  first  stage  we  have 
roughly  outlined  four  regions  having  densities  0.6,  0.75,  0.8,  and 
1.1.  According  to  our  definition,  the  center  pixel  with  value  0.  8  is  a 
boundary  pixel  and  has  to  be  reexamined.  By  redigitizing  with  a 
smaller  aperture  of  1/4  of  the  size  originally  used,  we  obtain  4  new 
readings  corresponding  to  this  one  pixel  as  shown  in  Fig.  4. 4- lb. 

We  replace  the  pixel  with  these  newly  obtained  data  as  shown  in 
Fig.  4.4-lc.  Now  once  more  we  have  to  decide  to  which  one  of  the 
regions  these  new  pixels  should  be  assigned.  At  this  point  an 
interesting  point  can  be  raised.  Equation  (4.2-8)  points  out  that  the 
new  data  obtained  by  redigitizing  the  photograph  is  twice  as  noisy  as 


In  practice  there  is  no  need  to  physically  scan  the  image  twice, 
once  with  each  aperture.  It  suffices  to  scan  the  image  with  the 
smaller  of  the  two  apertures.  Then  the  computer  can  be  used  to 
perform  a  2  X  2  averaging  on  the  digital  image  to  effectively 
obtain  what  we  would  have  obtained  had  we  actually  scanned  the 
image  a  second  time  with  the  larger  aperture. 
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A  boundry  pixel 
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(d)  Final  result 


Figure  4.4-1.  Second  stage  of  the  detection. 


the  data  which  was  used  originally  in  connection  with  the  first  stage 
of  the  detection.  This  occurs  because  the  scanning  aperture  is 
reduced  in  size  by  1/4.  Therefore  a  logical  question  is  to  ask  how 
the  noisier  data  can  be  used  to  sharpen  the  boundaries  detected 
using  data  less  noisy. 

One  possible  answer  is  now  described.  In  the  first  stage,  after 
observing  each  pixel  in  the  degraded  digitized  image,  we  had  to 
decide  to  which  one  of  M  regions  the  pixel  should  be  assigned.  In 
the  second  stage,  even  chough  we  are  using  noisier  data,  we  no 
longer  need  to  consider  all  of  the  M  regions  in  order  so  assign  the 
small  pixels  to  the  correct  value.  We  need  only  consider  the 
immediate  vicinity.  This  point  can  be  made  clear  by  the  example  of 
Fig.  4.4-1.  In  Fig.  4.4-lc  there  are  four  new  observations  having 
values  1.0,  0.89,  0.7  and  0.9.  Note  that  after  the  first  stage  we 
have  already  made  a  determination  that  the  area  these  4  pixels 
occupy  lies  at  the  boundary  of  the  two  extended  regions  having  values 
1.  1  and  0.8.  Therefore  with  these  new  four  data  points  we  have  only 
to  decide  to  which  one  of  the  two  regions,  1.  1  or  0.8,  we  should 
assign  the  new  pixels.  This  is  in  contrast  to  the  first  stage  where 
wc  decide  between  M  regions  in  assigning  each  pixel.  In  other 
words,  in  the  second  stage  we  do  conditional  detection  in  the  sense 
that  in  reassigning  each  boundary  point  we  consider  only  the  regions 
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on  whose  boundary  the  point  lies. 

The  actual  reassignment  can  be  done  according  to  different 
criteria.  One  possible  choice  is  according  to  the  absolute  value  of 
the  difference.  For  example  in  Fig.  4.4-lc  of  the  4  new  data  points, 
pixels  having  0.89,  0.7  and  0.9  are  assigned  to  the  region  having 
density  0.  8  while  the  fourth  pixel  having  value  1. 0  is  assigned  to  the 
region  density  1.  1.  Comparing  Fig.  4.4-la  and  4.4-ld  we  see  how 
the  data  from  the  second  digitization  of  the  photograph  is  used  to 
modify  and  sharpen  the  boundaries  between  regions  having  densities 
0.  8  and  1.1.  Similarly  all  other  boundary  points  should  be  reworked. 

This  finishes  explanation  of  the  basic  structure  of  the  algorithm. 
However,  if  the  complete  algorithm  is  to  be  useful,  in  practice,  some 
of  the  unrealistic  assumptions  that  were  made  in  the  derivation  must 
now  be  dropped.  This  is  done  in  the  next  section. 

4.  5  Detection  with  Unknown  Signal  Levels 

In  the  previous  analysis  we  assume  that  the  degraded  image 
consisted  of  exactly  M  regions  with  uniform  densities,  even  though 
the  regions  could  not  be  identified  exactly.  We  further  assumed  that 
the  density  levels  of  these  regions  were  known  a  priori.  Clearly 
these  assumptions  are  not  realistic.  In  this  section  we  intent  to  omit 
these  assumptions  and  reexamine  the  detection  algorithm  discussed 
earlier  in  the  chapter. 

We  will  retain  the  basic  two  stage  structure  of  the  detection 
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algorithm.  In  the  first  stage  we  use  a  quantizer  to  implement  the 
detector  and  use  as  observations  the  data  obtained  by  scanning  the 
film  with  an  aperture  of  size  2a  x  2a.  The  second  stage  is  not 
altered  because  all  the  assumptions  that  we  are  going  to  relax  affect 
the  first  stage  only.  Recall  that  the  quantization  levels  of  the  quantiz¬ 
er  used  in  the  first  stage  were  set  at  density  levels  D. ,  i  =  1 ,  . . .  ,  M 
which  were  assumed  known.  Since  we  no  longer  are  going  to  make 
this  assumption,  we  need  to  restructure  the  quantizer  to  be  used  in 
the  first  stage. 

As  before,  we  assume  that  the  images  are  composed  of  several 
extended  regions  each  having  a  uniform  density.  We  assume  that 
the  number  of  these  regions  and  their  density  levels  is  not  known, 
however  we  do  assume  that  the  image  has  very  low  contrast.  We 
examine  low  contrast  images  because  they  are  particularly  vulnerable 
to  film-grain  noise. 

To  construct  a  quantizer  for  use  in  the  first  stage,  we  begin  by 
arbitrarily  choosing  a  quantization  level  Q^.  Having  set  this  quantiza 
tion  level,  we  place  other  quantization  levels  above  and  below  such 
that  they  are  exactly  four  noise  standard  deviations  apart.  Since  the 
image  has  low  contrast,  one  can  approximate  the  standard  deviation 


of  the  noise  by 
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where  the  density  D  can  be  taken  to  be  the  sample  mean  taken  over  the 
entire  degraded  digitized  image. 

Let  the  quantization  levels  thus  obtained  be  denoted  by 

QL’  *  *  *  ’Q-2’  Q-1’Q0’  Ql'^2'  *  *  *  ,QU 
where  the  difference  between  each  successive  pair  is  4a.  The  actual 
number  of  quantization  levels  depends  on  the  expected  dynamic 
range  of  the  image  density.  As  discussed  in  Chapter  2,  for  almost 
all  classes  of  images  the  density  will  vary  in  the  range  of  0.0  to 
2.  5,  therefore  we  arbitrarily  select  somewhere  in  this  range  and 
then  place  other  quantization  levels  4a  apart  until  this  range  is  filled 
up. 

Having  set  the  quantization  levels,  we  set  the  decision  levels  at 
the  mid-point  between  each  pair  of  quantization  levels.  Thus  all  the 
decision  bands  are  40  wide.  We  have  now  constructed  a  quantizer 
without  any  a^  priori  knowledge. 

Now  we  investigate  the  consequence  of  utilizing  this  quantizer  in 
the  first  stage  of  the  detection  algorithm  discussed  earlier  and 
following  it  with  the  second  stage  which  is  left  unchanged. 

As  an  example,  suppose  we  digitize  a  small  section  of  a  photo¬ 
graph  by  scanning  it  with  a  very  small  square  aperture  of  size  2ax2a. 

The  digitized  image  will  be  degraded  with  a  film-grain  noise  having 
standard  deviation  a  as  given  by  eq.  (4.  5-1).  Suppose  the  section 
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of  the  photograph  that  we  have  scanned  is  composed  of  four  extended 


regions  R^.R^,  R^  and  R^  with  unknown  boundaries  and  unknown 
uniform  densities  and  D^.  If  the  image  has  low  contrast, 

these  four  regions  will  be  indistinguishable  from  one  another  in  the 
digitized  image  due  to  the  small  size  of  the  scanning  aperture  and 
enhancement  is  necessary. 

We  now  apply  the  two  stage  detection  algorithm  using  the 
quantizer  discussed  in  this  section  in  the  first  stage.  To  fully 
understand  the  nature  of  this  algorithm  let  us  assume  that  in  construc¬ 
ting  the  quantizer,  it  just  happens  that  ,  the  density  level  of  region 
Rj,  coincides  exactly  with  quantization  level  Q^.  Assume  also  that 
D^,  the  density  level  of  region  R^,  happens  to  be  one  0  greater  than 
quantization  level  and  that  and  happen  to  be  2 a  and  3o 
greater  than  respectively,  as  shown  in  Fig.  4.5-1. 

Using  this  quantizer,  all  pixels  in  the  degraded  image  having 
values  between  and  1  will  be  quantized  to  Q^.  On  the  other 
hand  we  know  that  pixels  in  the  degraded  image  belonging  to  region 
R^  are  Gaussian  distributed  with  mean  and  standard  deviation  o  . 
Therefore,  since  Dj  coincides  with  and  since  and  j  are  2o 
to  each  side  of  Q^,  then  upon  quantizing  the  degraded  image  95%  of 
pixels  belonging  to  region  Rj  will  be  quantized  to  Dj  while  2.5%  will 
be  quantized  to  Q^+|  and  another  2.  5%  will  be  quantized  to 
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Thus  after  the  first  stage  only  5%  of  the  pixels  belonging  to  region 
are  erroneously  assigned  to  the  wrong  density. 

As  outlined  before,  we  remove  all  isolated  points  after  quantiza¬ 
tion  and  prior  to  the  second  stage.  For  a  pixel  in  region  to  be  an 
isolated  point  after  the  first  stage,  it  should  be  an  erroneously  detected 
pixel  surrounded  by  8  pixels  which  have  correctly  been  detected  to 
have  value  D^.  Thus  the  probability  of  a  pixel  being  an  isolated  point 

g 

is  (.95)  or  roughly  0.66,  Thus,  by  removing  all  isolated  points 
from  this  region,  of  the  original  5%  falsely  detected  pixels, 0.  66x5%  = 
3.3%  are  corrected  to  have  value  leaving  only  1.7%  misclassified 
pixels  in  the  interior  of  this  region  prior  to  second  stage  of  the 
detection. 

In  the  second  stage  of  processing  the  edge  pixels  are  reworked. 
Because  1.7%  of  the  pixels  have  values  other  than  Dj  within  the 
interior  of  region  Rj,  then  all  these  pixels  and  their  8  closest 
neighbors  will  qualify  as  edge  pixels.  Therefore  9  x  1,7  or  about 
15%  of  the  pixels  in  region  will  be  redetected  using  data  obtained 
by  scanning  the  photograph  a  second  time  with  smaller  aperture  size 
axa.  Of  the  15%  reworked  pixels  some  will  ultimately  be  redetected 
to  have  value  Dj  thus  bringing  the  probability  of  correct  detection  in 
region  Rj  to  well  over  0.  85. 

The  above  completes  analysis  of  the  performance  of  the  detection 
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algorithm  for  regions  having  a  density  which  just  happens  to  coincide 

with  a  quantization  level  of  the  quantizer  used  in  the  first  stage  of 

the  algorithm.  However,  since  the  quantization  levels  are  placed  quite 

arbitrarily,  it  is  not  very  likely  that  many  regions  in  the  image  will 

have  a  density  that  will  coincide  exactly  with  a  quantization  level. 

Thus  it  is  more  interesting  to  consider  the  fate  of  regions  R^,  R^  and 

R  which  have  densities  that  fall  between  two  quantization  levels 
4 

rather  than  on  any  particular  one. 

We  first  consider  region  R2>  This  region  has  a  density  level 
which  exceeds  quantization  level  by  one  o  and  is  less  than 
quantization  Qk+1  bY  3o*  Since  degraded  digitized  image  pixels 
belonging  to  region  R^  are  Gaussian  distributed  about  mean  with 
standard  deviation  a,  and  since  decision  levels  1  and  of  the 
quantizer  are  3o  and  one  n  away  from  D2  respectively,  then  upon 
quantization,  84%  of  the  pixels  in  R2  will  be  quantized  to  while 
16%  will  be  quantized  to  Q^  +  j  • 

g 

After  quantization  a  pixel,  quantized  to  has  a  (*84)  or  .25 

probability  of  being  an  isolated  point.  Meanwhile  a  pixel  quantized  to 

O 

Q  has  (.  16)  or  about  zero  probability  of  being  an  isolated  point, 
k 

Thus  in  the  process  of  removing  isolated  points,  .25  x  16%  or  about 

4%  of  pixels  originally  quantized  to  Q^j  be  changed  to  Q^. 

Therefore,  prior  to  the  second  stage  of  the  detection,  88%  of  the 

interior  points  of  region  R-  are  given  value  Q,  while  the  remaining 
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12%  are  given  value  Q^j. 

In  reexamining  the  boundary  pixels  in  the  second  stage  of  the 
detection,  the  12%  pixels  having  value  +  j  and  their  8  closest 
neighboring  pixels  will  be  considered  boundary  pixels.  Thus 
9  x  12%  =  98%  or  roughly  the  entire  region  will  be  reexamined  in 
the  second  stage. 

Recall  that  in  the  second  stage  we  use  data  obtained  by  scan¬ 
ning  the  film  with  a  smaller  aperture  of  size  a  X  a  to  rework  the  bound¬ 
ary  pixels.  Clearly  in  region  R^  this  new  set  of  data  will  once  again 
be  Gaussian  distributed  with  mean  but  now  with  standard  deviation 
a  •  which  is  twice  as  large  as  a  from  eq.  (4.5-1). 

Since  pixels  in  region  R^  after  the  first  stage  of  detection  have 

values  or  Q^j,  l'16  newly  obtained  data  will  be  assigned  value 

Q,  or  Q,  ,  depending  on  the  absolute  value  of  the  difference 
k  k  +  1 

between  the  observed  data  and  these  two  values  as  explained  in 

section  4.4.  Referring  to  Fig.  4.  5-1  we  note  that  in  the  new  set  of 

data,  pixels  with  a  value  greater  than  will  be  closer  to  Q^+j 

while  pixels  with  a  value  less  than  will  be  closer  to  Q^.  Now 

since  is  n'/Z  away  from  D^,  from  Gaussian  distribution  properties 

it  is  clear  that  31%  of  pixels  in  the  new  data  will  be  greater  than 

and  thus  will  be  quantized  to  while  the  other  69%  will  be 

assigned  value  Q^.  Thus  in  the  final  detected  image,  region  R^  will 

be  composed  of  pixels  of  which  69%  have  value  Q  and  31%  have  value 
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Following  the  same  line  of  argument  we  used  to  analyze  regions 

Rj  and  R^,  one  can  show  that  in  the  final  detected  image  region  R^ 

will  consist  of  pixels  having  value  50%  of  the  time  and  value 

the  remaining  50%.  Region  R^  will  then  be  composed  69%  of  pixels 

with  value  Q,  ,  and  31%  with  value  Q,  .  Thus  various  regions  will  be 
kf  1  k 

coded  into  various  percentages  of  quantization  levels. 

4.6  Effectiveness  of  the  Algorithm  for  Enhancement 

At  first  one  might  question  the  usefulness  of  this  detection 

algorithm.  In  the  case  just  discussed,  we  detect  in  region  R^  about 

50%  of  the  pixels  to  have  value  and  the  other  50%  to  have  value 

while  in  reality  all  the  pixels  in  this  region  have  a  value  exactly 

half  the  distance  between  and  Thus  the  probability  of  error 

in  this  region  is  1.  However  in  image  detection  involving  the  human 

observer  the  primary  goal  is  distinction  between  regions  of  different 

density  levels  rather  than  the  density  value  itself.  Therefore  the 

conventional  detection  error  probability  is  not  indicative  of  the 

performance  of  this  detector. 

To  see  that  this  algorithm  accomplishes  the  goal  of  image 

detection,  consider  the  case  just  discussed  in  which  region  R^  is 

surrounded  by  region  R^  as  shown  in  Fig.  4.6-la.  Since  the  image 

pixels  in  region  Rj  are  detected  to  have  value  about  90%  of  the 

time  while  pixels  in  region  R,  are  detected  to  have  value  Q,  50%  of  the 
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The  cose  where  D|  foils  one  a  below  Qk 


Figure  4.6-1.  Composition  of  regions  in  two  different 
cases. 
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time  and  value  Q^  +  j  the  other  50%,  the  two  regions  will  be  easily 
distinguishable  from  one  another  in  the  detected  image.  This  is  due 
to  the  fact  that  human  eye  can  easily  separate  regions  having  such  a 
significant  difference  in  structure.  The  experimental  results  of  the 
next  chapter  will  bear  this  out. 

Referring  to  Fig.  4.5-1  we  see  that  the  density  levels  of  regions 
Rj  and  R^  are  2n  apart.  To  better  understand  the  algorithm,  suppose 
we  construct  our  quantizer  so  that  happens  to  fall  one  a  below 
quantization  level  rather  than  on  as  assumed  earlier,  and  that 
falls  one  n  above  Q^.  In  the  final  detected  image  we  will  find 
that  region  R^  is  detected  as  having  value  69%  of  the  time  and 
value  ^  31%  of  the  time,  while  in  region  R^  the  percentages  are  69% 
and  31%  j.  In  this  case  both  regions  have  value  69%  of 
the  time  and  the  eye  will  not  be  able  to  distinguish  these  two  regions 
as  easily  as  before  because  their  structures  are  not  sufficiently 
different. 

The  structures  of  these  and  all  other  regions  in  the  final  detected 

image  can  be  changed  by  shifting  the  quantization  levels  by  a  small 

amount.  In  section  4.4  we  initially  selected  quantization  level 

arbitrarily  and  then  placed  other  levels  such  that  adjacent  pairs  were 

4*7  apart.  As  discussed  in  the  example  of  Fig.  4.6-lb  the  mean 

densities  of  various  regions  relative  to  these  quantization  levels  in 

the  image  might  fall  in  locations  so  that  some  adjacent  regions  would 
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get  coded  into  a  very  similar  percentage  of  quantization  levels  and 
thus  not  he  sufficiently  distinguishable  from  one  another  in  the  final 


detected  image. 

If  we  shift  the  original  quantization  level  by  one  a  this  means 
that  all  quantization  levels  are  shifted  by  one  a  and  the  location  of 
mean  densities  of  various  regions  changes  relative  to  these  new 
quantization  levels.  Since  it  is  the  relative  distance  between  the 
quantization  levels  and  the  mean  densities  that  determines  how  regions 
are  coded  into  various  percentages  of  the  quantization  levels,  then 
the  appearance  of  the  final  detected  image  varies  substantially  as 
the  quantization  levels  are  shifted.  One  can  repeatedly  shift  the 
quantization  levels  by  small  intervals  until  the  human  observer  is 
satisfied  with  the  final  result. 

In  the  discussion  above  we  have  allowed  the  human  observer  to 
be  in  a  feedback  loop  in  order  to  improve  the  final  results.  Naturally 
there  are  some  cases  where  image  restoration  techniques  have  to  be 
designed  so  that  they  are  capable  of  processing  hundreds  of  degraded 
images  in  a  very  short  time.  In  such  cases  one  obviously  cannot 
insert  the  human  observer  in  the  feedback  loop  because  of  insufficient 
time  for  individual  inspection  of  each  restored  image.  However 
such  volume  processing  is  seldom  encountered  when  the  primary 
goal  of  restoration  is  the  elimination  of  film- grain  noise. 

Before  concluding  this  chapter  we  should  give  some  insight  into 
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selection  of  the  aperture  for  the  scanner.  The  selection  of 
appropriate  size  aperture  is  of  utmost  importance  and  there  are  two 
main  factors  to  consider.  The  most  obvious  one  is  the  size  of  the 


smallest  object  one  expects  to  find  in  the  section  of  the  film  being 

2 

scanned.  If  one  is  looking  for  objects  of  size  40  (um. )  the  size  of 

2 

the  scanner  aperture  must  be  considerably  smaller  than  40  (mu.  ) 

2 

more  likely  on  the  order  of  4  (jjm. )  .  This  is  done  so  that  the  scanner 
will  not  average  over  a  large  portion  of  the  object. 

The  second  factor  to  be  considered  is  the  contrast  of  the  adjacent 
regions.  If  we  hope  to  distinguish  two  adjacent  regions  whose 
densities  are  different  by  AD,  we  should  select  our  aperture  such  that 
the  resulting  noise  standard  deviation  is  no  more  than  AD  '2.  to  insure 
that  the  density  levels  of  these  two  regions  are  at  least  2 n  apart  and 
therefore  upon  quantization  they  are  not  often  assigned  to  the  same 
quantization  level  in  the  final  detected  image. 
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Chapter  5 


DETECTION  II 

In  Chapter  4  we  used  the  fundamentals  of  detection  theory  to 
provide  a  framework  to  analyze  the  "noise  cheating"  algorithm.  In 
this  chapter  we  use  the  same  framework  to  develop  more  versatile 
algorithms. 

In  the  last  chapter  we  considered  a  class  of  low  contrast  images 
with  small  dy‘ -imic  range.  The  assumption  of  small  range  for  the 
optical  density  enabled  us  to  omit  the  signal-dependence  of  the  noise 
and  take  a  ,  the  standard  deviation  of  the  noise  to  be  invariant  over 
the  entire  image.  Many  images  have  a  large  dynamic  range  in 
density,  yet  have  regions  of  local  low  contrast.  In  such  cases, 
ignoring  the  signal-dependence  of  the  noise  affects  the  performance 
of  any  detection  scheme  adversely. 

We  start  this  chapter  by  modifying  the  detection  scheme  discussed 
in  the  last  chapter  to  obtain  a  detector  which  takes  into  account  the 
signal-dependence  and  thus  is  applicable  to  a  broader  class  of  images. 
In  the  second  half  of  this  chapter  we  develop  a  more  versatile 
Bayesian  detection  scheme  enabling  us  to  accommodate  any  a  priori 
knowledge  available. 
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5.1  A  Detector  for  Signal-Dependent  Noiae 

The  need  for  considering  signal-dependence  in  film-grain  noise 
restoration  of  images  with  dynamic  range,  becomes  clear  by  examin¬ 
ing  the  signal-to-noise  ratio  (SNR)  of  the  degraged  image. 

In  a  local  portion  of  an  image  when  the  mean  density  is  D  we  can 
define  signal  to  noise  ratio  to  be 

2 

SNR(D)  =  —  (5.1-1) 

CTD 

where  oD  is  standard  deviation  of  the  noise.  We  equivalently  can 
write 


SNR(D) 


D 


(XD1'3,2 


(5.1-2) 


Let  us  consider  a  Kodak  Pan-X  film  which  has  been  scanned  with 
2 

a  4  (am.  )  aperture.  The  scanning  constant  k  for  this  film  and  this 
size  aperture  is  roughly  0.6.  Now  suppose  that  there  are  two 
regions  in  the  digitized  image,  one  having  mean  density  of  0.2  and 
the  other  having  mean  density  of  2,2.  Calculating  the  SNR  for  these 
two  regions,  from  eq.  (5.  1-2)  we  have 


and 


1  4/3 

SNR(.  2)  =  - =  (.2)  = 

(.  6) 


SNR (2.  2)  =  (2.2)4/3 

(.  6) 


0.  319 


=  8.478  . 


(5.  1-3) 


(5.1-4) 
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Thus  in  this  case  SNR  for  the  two  regions  of  this  image  varies  by 
a  factor  of  26  and  there  is  a  definite  advantage  in  considering  the 
signal-dependence  in  processing  this  degraded  image. 

With  the  above  motivation,  we  now  describe  modifications  to  the 
"noise  cheating"  algorithm  discussed  in  the  last  chapter  in  order  to 
incorporate  the  signal-dependence  effects.  In  section  4.4  we  out¬ 
lined  a  procedure  for  the  construction  of  a  quantizer  utilized  in  the 
first  stage  of  the  two-stage  detection  scheme.  In  constructing  this 
quantizer  we  place  the  initial  quantization  level  arbitrarily.  The 
other  quantization  levels  are  then  uniformly  spaced  at  4o  intervals 
when  a  ,  the  standard  deviation  of  the  noise,  is  assumed  to  be  a 
constant  for  all  the  various  regions  in  the  image.  In  quantizing  the 
degraded  image  with  this  quantizer  we  can  make  the  following 
remarks: 

a)  If  a  region  in  the  degraded  image  has  a  density  whose  value 
coincides  with  a  particular  quantization  level,  then  roughly  90%  of 
the  pixels  in  this  region  are  correctly  detected  and  their  values 
changed  to  the  density  of  the  regions. 

b)  If  a  region  in  the  degraded  image  has  a  density  whose  value 
falls  in  between  two  quantization  levels,  then  after  quantization  all 
the  pixels  in  this  region  are  assigned  a  value  from  one  of  the  two 
quantization  levels. 

Both  of  the  above  statements  depend  on  the  condition  that  all  decision 


levels  for  the  quantizer  are  2 a  away  from  their  quantization  levels. 

Equation  (2.3-10)  points  out  that  a,  the  standard  deviation  of  the 
noise,  is  a  monotonically  increasing  function  of  the  mean  density. 

Thus  if  we  insist  on  placing  the  quantization  levels  of  the  quantizer 
4~  apart,  consistently,  then  after  arbitrarily  selecting  the  initial 
quantization  level  the  other  levels  should  be  spaced  successively 
wider  as  we  move  toward  high  density  values  and  successively 
smaller  as  we  move  toward  zero  density.  In  the  last  chapter  we 
were  able  to  place  the  quantization  levels  at  uniform  spacing  be¬ 
cause  we  assumed  that  the  images  of  interest  had  a  very  small 
dynamic  range  and  thus  a  varied  very  little  from  one  region  of  the 
image  to  another.  If  a  quantizer  with  uniformly  spaced  quantization 
levels  is  used  to  detect  an  image  with  a  large  dynamic  range,  the 
effects  will  be: 

a)  Adjacent  regions  in  the  degraded  image  having  very  small 
densities  that  are  not  substantially  different  will  be  detected  as  the 
same  region  because  the  distance  between  the  quantization  levels  is 
far  more  than  the  deviation  of  the  pixels  in  these  regions  from  their 
mean  densities. 

b)  Regions  with  high  densities  will  be  erroneously  quantized  to 
several  levels  because  in  the  degraded  image  some  pixels  in  these 
regions  deviate  from  the  density  of  the  region  by  more  than  the 
distance  between  the  decision  levels  of  the  quantizer.  This  will 
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cause  a  region  of  a  uniform  density  to  appear  in  the  detected  image 
as  a  region  having  several  densities. 


In  order  to  avoid  these  effects  we  must  make  sure  that  the 
quantization  levels  of  the  quantizer  are  consistently  4 0  apart,  where 
if  is  allowed  to  increase  with  the  mean  density  according  to  eq. 
(2.3-10).  Accordingly  we  decide  on  the  following  procedure  to 
construct  the  quantizer.  We  place  the  initial  quantization  level 
arbitrarily  at  Q^.  The  next  quantization  level  greater  than  is 
denoted  and  is  found  such  that 


Q,  -  Qn  =  2o  +  Zn. 


where 


and 


k'°o> 


1/3 


) 


1/3 


(5. 1-7) 


(5.1-8) 


(5. 1-9) 


Combining  eq.  (5. 1 -7)  with  eqs.  (5. 1 -8)  and  (5.  1 -9)  we  get 


Ql  -  ZMC^)1'3  -  Qq  -  2k(Q0)1/3  =  0 


(5.1-10) 


or  equivalently 

Qj  -  3aQj  +  (3a  +  8k3)Qj  -  a3  =  0 


(5.1-11) 


where 


a 


(5.1-12) 
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Since  Qq  is  known,  then  a  can  be  calculated  from  eqs.  (5.1-8)  and 
(5.1-12).  Evaluation  of  Qj  thus  involves  solution  of  the  third  order 
equation  given  by  (5.  1-11).  This  equation  has  a  deterministic 
solution  til  of  the  form 


where 


and 


and  where 


Q  j  =  A+B+a 


a  .  'pfTJF 

B  -  V  f 

«  ■(?)’*(!)* 

P  =  6aZ  -  8k2 
2 

q  =  - 8ak 


(5.  1-13) 

(5.1-14) 

(5.1-15) 

(5.1-16) 

(5.1-17) 

(5.1-18) 


After  Qj  is  found,  we  repeat  this  procedure  to  find  Q^.  and 
so  on  until  a  quantization  level  exceeds  the  upper  bound  of  the 
expected  dynamic  range  for  the  optical  density  of  the  image.  In 
most  cases,  this  is  less  than  2.  5. 

We  can  also  follow  the  same  procedure  to  find  quantization  levels 
less  than  Q^.  For  example,  Q  the  first  quantization  level  smaller 
than  Qq  can  be  calculated 


(5.1-19) 
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We  continue  finding  levels  Q  etc.  until  we  get  a  quantization 

level  which  is  smaller  than  zero.  Because  the  standard  deviation 
as  calculated  from 


o  *  MQJ 


(5.1-20) 


becomes  extremely  small  as  approaches  zero,  there  will  be  an 
infinite  number  of  closely  packed  quantization  levels  near  zoer.  To 
avoid  this,  we  may  decide  a  priori  that  below  a  certain  level,  say  0.  3, 
we  abandon  this  procedure  and  set  all  the  quantization  levels  at  a 
uniform  spacing  of  4<^,  where  a.  is  obtained  from  eq.  (5.  1-20)  and 
is  the  first  quantization  level  greater  than  or  equal  to  0.  3. 

Figure  5.1-1  shows  a  set  of  quantization  levels  obtained  accord¬ 
ing  to  this  procedure  with  the  scanning  constant  k  =  .  12  and  the  initial 
quantization  level  =  0.4. 

As  discussed  in  section  2  of  chapter  4,  we  would  like  to  find  a 

set  of  decision  levels  for  this  quantizer  corresponding  to  maximum 

likelihood  detection  of  regions  whose  mean  densities  happen  to 

coincide  with  one  of  the  quantization  levels.  Consider  decision  level 

L.  which  is  to  be  placed  in  between  quantization  levels  Q,  and  C,  , . 
k  k  kf  l 

Any  two  regions  in  the  degraded  image  having  mean  densities  equal 

to  and  will  have  pixels  which  are  Gaussian  distributed 

about  mean  Q,  and  Q  .  with  standard  deviations  and  as 

k+  'k  ^k+l 

given  by  eq.  (5. 1-20).  The  graphs  of  these  two  Gaussian  probability 
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Q  +2or  =1.5430 
2  Qa 


1 


distribution  functions  will  intersect  one  another  at  a  point  h  where 

aQk+i  *  Qk  ‘  °Q  *  Qk+i  ±0Qk°Qk+1  *  A 
h=  - 2 - —2 - ^ -  (5.1-21) 

aQ  "  °Q 

k+1  k 

and  where 


Q, 


A  =  ./<Q  k+rQk>  -21n(^—  X"o.  ,  "aQ,  )  *  (5*1-22> 


‘k+1 


k+1 


Note  that  A  is  always  positive,  so  that  h  will  always  have  two  real 

solutions  of  which  only  one  lies  between  Q,  and  Q,  . .  A  Gaussian 

k  k+1 

2 

probability  density  with  mean  Q,  .  and  variance  n_  lies  above  a 

“ktl 

Gaussian  distribution  with  mean  and  variance  for  all  points 

K 

greater  than  h  and  will  lie  below  this  function  for  all  points  less  than 
h.  Accordingly,  we  set  the  decision  level  at  h  so  that  quantization 
with  this  quantizer  corresponds  with  maximum  likelihood  detection 
in  the  sense  described  earlier.  Figure  5.  1-2  shows  a  pair  of 
quantization  levels  and  their  corresponding  decision  level  obtained 
according  to  the  above  procedure. 

The  two-stage  detection  algorithm  now  involves  quantization  of 
the  degraded  image  with  a  quantizer  constructed  according  to  the 
procedure  outlined  in  this  section.  As  before,  the  rough  boundaries 
of  various  regions  in  the  image  are  detected.  This  is  followed  by  a 
second  stage  of  detection  which  is  exactly  the  same  as  that  described 


in  section  4.  4. 


J - Qm|=  .7996 

2a0  =.2228 

Uk*l 

- Lk  =  .5825 


2crk  =  .1768 

i - Qk  =  .4 


Figure  5.  1-2.  An  example  of  placement  of  a  reconstruction 

level  which  corresponds  to  maximum  likelihood 
detection. 
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This  algorithm  was  applied  to  degraded  images  and  some  exper¬ 


imental  results  are  presented  at  the  end  of  this  chapter. 

5.  2  Bayesian  Detector 

In  the  last  section  we  discussed  an  algorithm  which  performs 
a  form  of  suboptimal  maximum  likelihood  detection  using  a 
quantizer  with  nonuniformly  spaced  levels.  In  this  section  we  start 
with  the  same  nonuniformly  spaced  levels  except  we  now  develop  a 
more  flexible  Bayesian  detector  to  replace  the  quantizer. 

To  begin,  briefly  reivew  M-hypothesis  Bayesian  detection  from 
classical  statistics.  A  detection  model  is  usually  composed  of  the 
four  separate  spaces  as  shown  in  Fig.  5.2-1. 

I)  Space  ft .  This  is  the  set  of  all  possible  signals,  and  may  be 
continuous  or  discrete.  Elements  of  this  space  are  denoted  by  D. 

The  a  priori  probability  distribution  over  the  signal  space  is  given  by 
f  .(D),  so  that 

J  f  (D)dD  =  1  .  (5.2-1) 

ft 

II)  Space  V.  This  is  the  set  of  all  observabled.  In  the  termin¬ 
ology  of  communication  theory,  a  transmitted  signal  Deft  becomes  an 
observation  veV  due  to  the  noise.  This  space  is  described  by  a 
conditional  probability  density  f^(v|D),  which  is  the  probability  of 
observing  v,  given  a  particular  signal  D.  This  density  also  satisfies 


t 
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r  f  (v|D)dv  =  1.  (5.2-2) 

V  v 

III)  Space  V.  This  is  the  hypothesis  space.  Elements  of  U 

are  various  hypotheses  regarding  which  signal  D  was  transmitted.  A 
particular  observation  v  leads  to  the  selection  of  an  HeV. 

IV)  Space  /?.  This  is  the  action  space.  In  general  we  may 
observe  a  v  and  instead  of  deciding  on  a  particular  hypothesis 
H.cV  ,  we  may  decide  on  several  hypotheses  but  with  a  probability 
distribution  assigned  to  them.  Elements  of  the  space  f?  are 
probability  distribution  functions  and  are  denoted  by  r(H|v).  They 
denote  the  probability  distribution  over  the  space  V  when  v  is 
observed.  Elements  of  %  are  called  rules. 

In  addition  to  the  above  four  spaces,  there  are  usually  three 
other  notions  that  complete  the  detection  model.  They  are: 

i)  Cost.  This  is  denoted  by  C(D,  H).  It  is  a  real  valued  function 
defined  on  space  3xV  which  gives  the  cost  of  deciding  on  hypothesis 
H  when  signal  D  was  transmitted. 

ii)  Risk.  This  is  denoted  by  R(D,  r)  and  is  defined  by 

R(D,r)  =  f  [  C(D,H)f  (vlD)r(H|v)dHdv.  (5.2-3) 

V  *  V 

This  expression  describes  the  average  cost  when  a  particular  signal 
D  is  transmi'  rd  and  we  use  rule  r  to  decide  on  our  hypothesis. 

iii)  Average  risk.  This  is  described  by 
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(5.2-4) 


R(r)  4  J  R(D,  r)f^D)dD 

and  denotes  the  average  risk  associated  with  selecting  a  particular 

rule  r.  The  averaging  is  performed  over  the  signal  space. 

The  Bayesian  detector  is  a  particular  detector  that  observes  v 

and  decides  on  a  rule  r  such  that  the  average  risk  R(r)  is  minimized. 

A  special  case  of  this  model  is  when  we  limit  the  hypothesis 

space  V  to  only  M  elements  H, , . . .  ,  Hw.  In  this  case  we  divide  the 

1  M 

signal  space  into  M  regions  R Rw  as  shown  in  Fig.  5.2-2. 
Hypothesis  then  corresponds  to  the  assumption  that  the  transmitted 
signal  is  from  region  R..  Denoting  by  P(R.)  the  a  priori  probability 
that  a  signal  might  belong  to  region  R.^  and  by  f^(D|R.)  the  a  priori 
conditional  probability  of  the  signal  within  region  R.  we  have 

M 

=  g  P(R.)fyD|R.)  . 

One  can  show  that  in  the  case  the  minimization  of  the  average 
risk  results  in  the  following  detection  procedure  for  a  Bayesian 
detector.  We  calculate 


(5.2-5) 


A.(v)  =  2_{  C(D.H.)fv(v|D)f^D|R.)dD  (5.2-6) 


R. 

i 


i  =  1. 


M 


then  we  announce  hypothesis  as  being  true  if 
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A.  (v)  <A.(v)  i  =  .  (5.2-7) 

k  i 

i  i  k 


Some  simplifications  can  be  made  if  we  use  the  cost  function 


C(D,  H  ) 


0  if  DeR. 

3 

1  if  D^R. 

3 


(5.2-8) 


This  is  a  very  reasonable  cost  function  which  does  not  penalize 
correct  detection.  For  this  cost  function  eq.  (5.2-6)  reduces  to 


M 

P(R. )  [  f.,(v |  D)f  (D  |  R.  )dD  (5.2-9) 

i  JR  V  3  i 

i 

From  eq.  (5.2-9)  it  is  obvious  that  the  test  rule  described  by 
eq.  (5.2-7)  can  equivalently  be  Stated  as:  for  each  observation  v 
announce  hypothesis  as  being  true  if 

B  (v)  >  B.(v)  i  =  1 . M  (5.2-10) 

K  l 

i  ^  k 

where 

B.(v)  =  P(R.)  [  f  (vlD)f  (D|R.)dD  .  (5.2-11) 

i  i  JR  V  3  i 

i 

With  this  background  we  now  apply  the  Bayesian  detection  theory 
to  our  previous  problem.  Suppose  we  decide  on  M  nonuniformly 
spaced  levels  in  the  density  range  of  say  [o,  0,  2.  5l  as  discussed  in 
the  last  section.  These  levels  are  selected  so  that  the  distance 


A.(v)  = 
3 


■s 
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between  them  increases  as  we  move  up  in  the  interval  [0.  0,  2.  5]  . 

Let  us  denote  these  levels  by  Dj, . . . ,  D  .  *  The  detection  algorithm 
discussed  in  section  5.  1  involves  maximum  likelihood  detection  of 
regions  in  the  degraded  image  having  mean  densities  coinciding  with 
one  of  these  levels.  The  detection  was  implemented  by  using  a 
quantizer  in  the  first  stage  and  followed  by  a  second  stage  as  dis¬ 
cussed  in  section  4.4.  Keeping  the  second  stage  of  the  detection 
exactly  the  same,  we  now  change  the  first  stage  from  maximum 
likelihood  detection  to  Bayesian  detection.  In  order  to  do  this  we 
start  by  defining  regions  Rj , .  . . ,  corresponding  to  the  model 

of  Fig.  5.2-2.  Referring  to  Fig.  5»  2-3  we  let  D. , . . . ,  D_  .  be  the 

1  M 

nonuniformly  spaced  levels  discussed  earlier.  We  also  define  M 
intervals  of  width  A 

Ri  "  fDi  "  2'  °i  f  2  1  i  =  1 . M  (5.2-12) 

where  A  is  an  arbitrarily  small  number.  Now,  corresponding  to  the 
detection  model  of  Fig.  5.2-2,  let  the  signal  space  be 

M 

*  =  Ri  •  (5.2-13) 


In  the  last  section  we  labeled  these  levels  by  . . .  ,Q  Q  Q 
Q^,  ...  as  they  were  also  to  denote  quantization  levels  of  the 
quantizer  used  in  the  first  stage  of  the  detection  algorithm. 
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In  the  enhancement  problem  the  observable  is  the  degraded  image 
and  pixels  in  the  degraded  image  correspond  to  elements  of  observa¬ 
tion  space  V.  The  enhancement  procedure  involves  examining  each 
pixel  in  the  degraded  image  and  deciding,  according  to  eq.  (5.2-10) 
which  interval  R.  it  most  likely  belongs  to.  The  value  of  the 
observed  pixel  is  changed  to  if  it  is  decided  that  the  pixel 
belongs  to  region  R.. 

Corresponding  to  the  signal  space  £  defined  inneq.  (5.2-13) 


we  can  rewrite  eq.  (5.2-11)  as 


B.(v)  =  P(R.)  f 


d.  +  4 

i  2 


i'  l  A  V 

d.  4 

i  2 


fv(v  |  D)f ft(D  |  R  )dD 


(5.2-14) 


where  f.y.(v|  D)  in  the  above  equation  is  the  conditional  probability 
density  of  the  observable  v  given  that  the  signal  was  D.  As  noted 
in  Chapter  2,  when  the  observable  is  an  image  degraded  by  film- 
grain  noise  then  this  conditional  density  is  Gaussian  with  the  form 


.  -[(v-D)2/£oi)] 

fv<v,D)  =  id —  e 
V  JZrtt, 


(5.2-15) 


where 


ctd  =  k(D) 


(5.2-16) 


Thus  if  a  priori  probabilities  P(R^)  and  f^(DjR.)  are  known,  for 
each  observed  pixel  value  v  in  the  degraded  image,  one  can  evaluate 
the  following  integral 


( 


-t(v-D)2/(2k2D2/3)l 

•  f .  ( D  |  R .  )d  D  (5.2-17) 

Jj  \ 

for  i  =  1 ,  . .  . ,  M  and  then  according  to  the  detection  rule  stated  in 
eq.  (5.2-10)  change  the  value  of  the  pixel  to  one  of  the  M  levels 


Since  the  detection  procedure  is  to  be  carried  out  on  a  digital 
computer,  evaluation  of  the  integral  of  eq.  (5.2-17)  for  each  pixel 
of  the  image  becomes  quite  time  consuming.  Therefore  in  practical 
use,  the  following  simplifying  assumptions  are  made: 

1)  We  assume  that  if  the  signal  D  belongs  to  region  R.  then  it  is 
equally  likely  to  be  anywhere  in  that  interval.  Thus  we  let 

fft(D|R.)  =  .  (5.2-18) 

0  elsewhere 

2)  If  £  is  small,  one  can  approximate  in  eq.  (5.2-14)  by 

crD  =  °D  "  k<Di>1/3  (5.2-19) 

i 

for  all  O  belonging  to  R^.  That  is  we  neglect  the  signal  dependence 
of  the  noise  within  each  interval.  Note,  however,  that  we  allow  the 
variance  of  the  noise  to  vary  from  one  Interval  to  the  next. 

Incorporating  these  two  assumptions  in  eq.  (5.2-17)  we  get 
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P(R.)  °i+2 


B.(v)  =  — 
i  A 


M 


1 


-[(v-D)2)/(2k2D2/3)l 


,/Ztt  kD. 

D.-| 

i  c. 


TT3  e 


dD.  (5.2-20) 


Recognizing  the  integral  in  the  eq.  (5,2-20)  as  an  error  function 
(erf),  it  has  the  closed  form  solution 


P(R. ) 


B. (v)  = 


erf 


v  -  D  +■£ 
i  2 


V  '  D  -  ? 


( - " — ~  el"f( - '—) 

\  aD  )  V  aD  ) 


(5.2-21) 


L  l  1  J 

which  reduces  the  evaluation  of  the  integral  to  a  table  look  up. 

In  summary,  the  Bayesian  detection  algorithm  involves  the 
evaluation  of  B.(v)  for  i  =  1,,,,,M  of  eq.  (5.  2-21)  for  each  pixel  of 
the  degraded  image  and  subsequent  detection  according  to  the  rule 
stated  in  eq.  (5.2-10).  The  boundaries  in  the  Bayesian  detected 
image  are  then  sharpened  according  to  the  procedure  described  in 
section  4.  4. 

5.  3  Relation  of  the  Bayesian  Algorithm  to  the  Noise  Cheating 

Algorithm 

In  section  4.  5  we  noted  that  the  "noise  cheating"  algorithm 
suggests  the  use  of  the  human  observer  in  a  feedback  loop  to  shift 
the  quantization  levels  by  a  small  amount  to  obtain  different  detected 
images.  This  would  be  done  if  the  detected  image  had  poor  visual 
quality  for  a  particular  set  of  quantization  levels.  In  fact  this 
procedure  can  be  repeated  any  number  of  times  and  the  weighted  sum 
of  the  various  detected  images  can  be  taken  to  be  the  final  enhanced 
image.  In  this  section  we  show  that  under  certain  conditions. 


*4  : 


performing  the  Bayesian  algorithm  once  is  equivalent  to  performing 
the  "noise  cheating"  algorithm  many  times  by  shifting  the  quantiza¬ 


tion  levels  within  a  band  of  width  A . 

If  a  particular  choice 

fjp|R.)  =  6(D-D.)  (5.2-22) 

is  made  in  eq.  (5.2-11)  we  obtain 

B.(v)  =  P(R.)fv(v|D.)  (5.2-23) 

Furthermore,  if  we  allow  all  regions  in  the  signal  space  to  be 
equally  likely  (i.e.  P(Rj)  =  P(R£)...  =  P(R^)),  then  the  detection 
rule  of  eq.  (5.2-10)  reduces  to  deciding  on  hypothesis  if 

fy(v|Dk)  >  fv(v|D.)  i  =  l . M  (5.2-24) 

i  4  k 

which  is  the  same  as  the  maximum  likelihood  detection  rule  of  eq. 

(4. 1-2)  used  in  "noise  cheating"  algorithm.  We  can  now  observe 
that  choosing  the  a  priori  conditional  probability  density  function 
f^(D  | Ri )  be  a  general  probability  density  function  then  Bayesian 
algorithm  is  equivalent  to  a  "noise  cheating"  algorithm  in  which  we 
have  allowed  the  quantization  levels  to  vary  within  a  band  of  width  A 
about  D.  =  i  =  1,,..,M  and  where  all  the  detected  images  obtained  due 
to  this  shift  of  quantization  levels  are  weighted  by  f^(D|R.)  to  obtain 
the  final  enhanced  image. 


5.  4  Experimental  Results 


In  this  section  we  apply  the  different  detectors  discussed  in  this 
and  the  last  chapter  to  some  degraded  images  and  compare  their 
results. 

Figure  5.4-la  shows  a  simulated  block  U  used  as  the  ideal  image. 
The  density  levels  for  this  image  are  0.4  for  the  background,  1.72 
for  the  U  and  1.  65  for  the  open  slot  in  the  U.  This  image  of  size 
512  x  512  was  chosen  because  it  has  a  reasonably  large  dynamic 
range  (0.4  to  1.72)  and  regions  of  very  low  contrast  (the  difference 
between  the  density  of  the  U  and  the  open  slot  in  the  U  is  only  0.07). 

In  addition  this  image  is  composed  of  extended  regions  and  this  is 
of  the  type  for  which  the  algorithms  discussed  in  the  last  two 
chapters  are  most  useful. 

This  ideal  image  was  degraded  according  to  the  model  of 
Fig.  2.8-3  to  obtain  the  image  shown  in  Fig.  4.4-lb.  Figure  5, 4-lc 
shows  the  detected  image  using  the  "noise  cheating"  algorithm  with 
the  initial  quantization  level  set  at  0.4  and  the  other  levels  uniformly 
set  at  4o  spacing  where 

o  =  k(D)1/3  =  ,3(.4)1/3  .  (5.4-1) 

Although  some  of  the  noise  is  removed  by  the  algorithm,  the 
result  is  difficult  to  identify^  This  points  out  a  deficiency  of  the 
"noise  cheating"  algorithm  in  ignoring  the  signal-dependence  of  the 
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(a)  Ideal  image 


(b)  Degraded  image 


(c)  Detected  image  using  ''  noise 
cheating"  algorithm 


Figure  5.4-1.  Application  of  "  noise  cheating"  algorithm. 


(a)  Ideal  image 


(b)  Degraded  image 


(c)  Detected  image 


Figure  5.4-2.  Application  of  the  detection  algorithm  of 
s  ection  5. 1 . 


(a)  Ideal  image 


Figure  5.4-3.  Application  of  Bayesian  detection  algorithm 


noise  in  enhancement  of  images  with  large  dynamic  range. 

Figures  5.4-2c  and  5.4-3c  show  respectively  the  result  of 
applying  the  algorithm  developed  in  section  5. 1  and  the  Bayesian 
algorithm  to  the  same  degraded  image.  The  block  letter  U  is  now 
recognizable  in  these  pictures.  Comparing  Figs.  5.4-2c  and  5.4-3c 
with  Fig.  5.4-lc  one  sees  the  advantage  of  these  algorithms  over 
the  "noise  cheating"  scheme. 

Considering  the  extreme  degree  of  the  degradation,  these 
techniques  do  a  good  job  of  identifying  and  enhancing  the  image 
features.  It  seems  unlikely  that  conventional  linear  or  nonlinear 
estimation  techniques  can  produce  better  quality  enhanced  images 
than  those  shown  in  Fig.  5.4-2c  and  5.4-3c. 

We  also  displayed  the  degraded  image  on  the 
and  tried  to  adjust  the  brightness  and  the  contrast  knobs  to  see  if  the 
image  could  be  enhanced  so  that  the  letter  U  would  become  visible 
on  the  screen.  However  due  to  extreme  degree  of  degradation  no 
amount  of  adjustment  did  the  job. 
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Chapter  6 


CONCLUSIONS  AND  SUGGESTIONS  FOR 
FUTURE  RESEARCH 

This  dissertation  has  described  different  means  of  restoring 
images  degraded  by  film-grain  noise.  We  have  used  some  experi¬ 
mental  and  theoretical  results  by  photographic  scientist  to  augment 
existing  mathematical  models  for  imaging  systems  with  film  as  the 
recording  medium.  The  modeling  has  two  primary  objectives  which 
sometimes  are  conflicting.  The  first  objective  has  been  to  obtain 
a  mathematically  tractable  model  for  subsequent  estimation  and 
detection  of  the  degraded  observed  image.  The  second  objective 
has  been  to  achieve  an  accurate  model  for  the  physical  process.  A 
subjective  test  has  been  devised  to  check  for  the  accuracy  of  the 
model.  Chapter  2  of  this  dissertation  concludes  with  a  particular 
mathematical  model  which  is  a  reasonable  representation  of 
various  effects  of  imaging  process  without  being  oversimplified. 

However  more  work  in  the  area  of  modeling  remains  to  be 

done.  We  mentioned  in  Chapter  2  that  the  standard  deviation  of  the 

form  given  by  eq.  (2.  3-10)  is  not  consistant  with  the  assumption  of 

Gaussianity  of  the  noisy  for  high  and  low  density  values.  To  fully 

understand  the  nature  of  the  distributing  of  the  noise,  one  needs  to 

expose  many  samples  of  different  films  uniformly  and  digitize  them 
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with  a  microdensitometer  using  various  apertures.  The  success  of 
such  a  procedure  depends  greatly  on  the  ability  to  produce  truly 
uniformly  exposed  samples.  In  past  researchers  doing  such  studies 
have  used  only  500  to  1000  points  in  determining  the  statistics  of  the 
noise.  For  accurate  assessment  many  more  points  should  be 
considered  and  statistical  tests  should  be  applied  to  insure  that  the 
estimates  are  consistent  and  accurate.  With  the  improvement  of 
microdensitometers,  availability  of  digital  computers  and  improve¬ 
ment  in  instrumentation,  examining  uniformly  exposed  films  may 
finally  provide  some  conclusive  results  regarding  the  form  and  the 
distribution  of  film-grain  noise. 

Another  unexplored  area  in  which  fruitful  research  may  be 
conducted  is  the  study  of  the  sensitivity  of  human  vision  to  film- 
grain  noise  in  various  ranges  of  density.  For  example  one  can 
perform  experiments  similar  to  Weber's  experiment  [l  “]  to  deter¬ 
mine  the  just  noticeable  contrast  over  a  wide  range  of  densities 
when  the  image  is  degraded  by  the  signal-dependent  film-grain  noise. 
The  results  of  such  a  test  may  be  used  to  improve  imaging  system 
models  by  including  the  effects  of  the  human  visual  system  for 
enhancement  and  restoration. 

In  Chapter  3  we  have  presented  a  linear  estimation  technique 
which  accounts  for  signal-dependent  noise.  The  method  is  shown  to 
be  superior  to  conventional  Wiener  filtering.  This  area  can  be 
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further  explored  by  considering  nonlinear  estimation.  We  have 
obtained  the  general  problem  of  restoration  of  images  degraded  by 
film-grain  noise  in  the  framework  of  estimation  theory  and  have 
shown  the  optimal  solution  to  be  an  estimate  based  on  nonlinear 
observations.  The  optimal  estimate  based  on  nonlinear  observations 
is  still  an  unsolved  problem  in  the  field  of  estimation  theory.  The 
most  recent  work  in  this  area  is  due  to  Naraghi  ,  who  proposes 
a  suboptimal  recursive  estimate  for  nonlinear  observations.  An 
interesting  area  for  future  study  is  the  application  of  Naraghi's  work 
and  other  nonlinear  estimation  techniques  to  images  degraded  by 
film-grain  noise. 

In  Chapters  4  and  5  we  have  concentrated  on  the  detection  of  a 
class  of  low  contrast  images  degraded  by  severe  film-grain  noise. 

An  already  existing  algorithm  called  "noise  cheating"  has  been 
analyzed  in  the  context  of  detection  theory.  Other  algorithms  are 
developed  based  on  the  same  principle  which  are  applicable  to  a 
wider  class  of  images  and  are  more  versatile.  These  detectors 
produce  fairly  reasonable  results  when  other  conventional  restoration 
methods  are  ineffective  for  extremely  severe  noise. 

In  these  detection  schemes  each  individual  pixel  in  the  degraded 
image  is  detected  independent  of  the  neighboring  pixels.  A  future 
study  might  explore  the  possibility  of  detecting  a  pixel  based  on  the 
result  of  the  detection  of  the  surrounding  pixels  using  the  correlation 


among  the  pixels. 


From  the  results  of  this  study,  it  seems  certain  that  future  for 
image  restoration,  especially  with  signal-dependent  noise,  lies  in 
the  use  of  nonlinear  techniques.  It  still  remains  a  challenge  to  find 
efficient  algorithms  for  rapid,  high  quality  restoration  on  pictures 
with  only  a  moderate  amount  of  noise,  and  it  is  hoped  that  the  results 
presented  here  will  contribute  to  the  development  of  these  algorithms. 
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